CSV

You can read and write from CSVs using Polars.

Reading

You can connect to a CSV file, like the UK Census file ./data/large/census.csv, without bringing it in memory, with the LazyCsvReader. Run this code using cargo run -r --example 2_2_1_read_csv.

use polars::prelude::*;

// Connect to LazyFrame (no data is brought into memory)
let lf = LazyCsvReader::new(PlPath::from_str("./data/large/census.csv"))
    .with_has_header(true)
    .finish()
    .unwrap();

None of the data is brought into memory. You can't even visualize any of it, since LazyFrame does not implement display. To display it, you can subset it to a few rows and then convert it to a DataFrame for printing:

println!("{}", lf.limit(5).collect().unwrap());

shape: (5, 21)
┌─────────────────┬────────┬───────┬──────┬───┬─────┬───────────┬────────┬───────┐
│ id              ┆ social ┆ birth ┆ econ ┆ … ┆ sex ┆ keep_type ┆ income ┆ chunk │
│ ---             ┆ ---    ┆ ---   ┆ ---  ┆   ┆ --- ┆ ---       ┆ ---    ┆ ---   │
│ str             ┆ i64    ┆ i64   ┆ i64  ┆   ┆ i64 ┆ i64       ┆ i64    ┆ i64   │
╞═════════════════╪════════╪═══════╪══════╪═══╪═════╪═══════════╪════════╪═══════╡
│ PTS000000455411 ┆ 4      ┆ 1     ┆ -8   ┆ … ┆ 1   ┆ 1         ┆ 86437  ┆ 47    │
│ PTS000000173083 ┆ 2      ┆ 1     ┆ -8   ┆ … ┆ 1   ┆ 1         ┆ 14782  ┆ 47    │
│ PTS000000176031 ┆ 4      ┆ 1     ┆ -8   ┆ … ┆ 2   ┆ 1         ┆ 36486  ┆ 47    │
│ PTS000000176069 ┆ 2      ┆ 1     ┆ -8   ┆ … ┆ 2   ┆ 1         ┆ 92092  ┆ 47    │
│ PTS000000262855 ┆ 2      ┆ 1     ┆ -8   ┆ … ┆ 2   ┆ 1         ┆ 14583  ┆ 47    │
└─────────────────┴────────┴───────┴──────┴───┴─────┴───────────┴────────┴───────┘

Writing

You can write to CSV any DataFrame you have in memory. For this example, we will bring one percent of the UK Census into memory. Run this code using cargo run -r --example 2_2_2_write_csv.

use polars::prelude::*;

// Read `census_0.csv` as LazyFrame
let lf = LazyCsvReader::new(PlPath::from_str("./data/csv/census_0.csv"))
    .with_has_header(true)
    .finish()
    .unwrap();

// Bring it into memory (by converting it to DataFrame)
let mut df = lf.collect().unwrap();

In order to save it, you have to create a file and write to it:

// Write `census_0.csv`
let mut file = std::fs::File::create("./data/temp_data/census_0.csv").unwrap();
CsvWriter::new(&mut file).finish(&mut df).unwrap();