How to convert SAV to CSV in Python
Set the options below and copy a script that reads your .sav file and writes a CSV with the value labels still readable.
Short answer
- pyreadstat.read_sav() reads the file, including
.zsav, and hands back a dataframe and a metadata object. - It gives you numeric codes by default. Value labels need
apply_value_formats=True, which is the opposite of what R users expect. pandas.read_spss()is a wrapper around the same library with the inverted default, and it hands back only a dataframe.- Files larger than memory go through
read_file_in_chunks, one dataframe at a time.
The default that costs people an afternoon
Almost every report of a CSV full of ones and twos comes down to one argument. pyreadstat.read_sav has apply_value_formats=False as its default, so the dataframe carries the codes SPSS stored and to_csv faithfully writes them out. Nothing failed, nothing warned, and the file looks plausible until somebody tries to read a column.
What makes this specific to Python is the direction. In R, foreign::read.spss applies value labels unless you turn them off, so a researcher porting a working R recipe reaches for the same intent and gets the reverse result. The generator above makes the argument visible in the script rather than leaving it implied, which is the whole reason it emits apply_value_formats=True as its own token you can see and delete.
There is a second layer to it. With labels applied, formats_as_category defaults to True, so those columns arrive as pandas categoricals. That changes nothing in the CSV, since the text is written either way, but it does change how the column behaves if you keep working in pandas afterwards.
The metadata object is the part a CSV cannot carry
Calling pyreadstat directly returns two things, and the second one is usually ignored. meta.column_names_to_labels holds the question text behind each column, the sentence a respondent actually saw. meta.variable_value_labels holds the answer coding, variable by variable. meta.missing_ranges holds what the file declared as missing.
None of that survives the trip to CSV, because a CSV has one row of column names and nothing else. This is the strongest argument for calling pyreadstat rather than pandas.read_spss, which returns only a dataframe. Recent pandas versions do stash the metadata ondf.attrs, which is useful to know and not something to build on, since it is not part of what the function promises. Tick the codebook option above, on a single file, and the script writes the name to label mapping into a second file beside your data.
Missing values are three different things, then one
SPSS questionnaires routinely code refused as 97, does not apply as 98 and never asked as 99. Those are user-defined missing values, and pyreadstat flattens every one of them to NaN unless you pass user_missing=True. After that flattening there is no way to recover which was which, so if the distinction matters to your analysis it has to be preserved on the way in rather than reconstructed later.
Not ideal for: anyone who wants the codes and the missingness at once without thinking about it. Keeping user missing means those columns stay numeric with real values in them, so a mean over that column will quietly include 99 unless you handle it. The tool cannot decide that for you, and a page that pretended otherwise would be doing you harm.
When the file is larger than the machine
A .sav from a national survey can outrun a laptop. pyreadstat.read_file_in_chunks takes a reader function and a chunk size and returns a generator, so you hold one chunk at a time. The script above writes the header with the first chunk and appends the rest, which is the part people get wrong when they write it themselves and end up with the column names repeated every ten thousand rows.
Two things change once you are chunking. Each chunk carries its own metadata object, so anything you want to say about the file as a whole has to come from the first one or from a metadata-only read. And anything that needs a whole column at once, a global sort or a total, belongs after the CSV is finished rather than inside the loop.
There is a multiprocessing route as well, both as a flag on read_file_in_chunks and as read_file_multiprocessing for a whole file. Both trade memory for speed, so on a file that already did not fit, reach for plain chunking first.
The five steps, without the tool
1. Install the reader
pip install pyreadstat. pandas.read_spss needs it too, so this step is not optional either way.
2. Read the file, and decide about labels while you do
df, meta = pyreadstat.read_sav("data/survey.sav", apply_value_formats=True). Without that argument the dataframe holds numeric codes, because False is the default.
3. Check what the metadata says the columns are
meta.column_names_to_labels gives the question text behind each column, and meta.variable_value_labels lists which variables carry answer labels at all. This is the fastest way to see whether the label step did anything.
4. Write the CSV
df.to_csv("data/survey.csv", index=False). Add encoding="utf-8-sig" if the file is going to be opened in Excel on Windows.
5. Open the result and check one coded column
Look at a variable you know is coded. If it reads Male rather than 1, the label step worked and the rest of the file almost certainly did too.
Questions people hit on the way
Why does my CSV show 1 and 2 instead of Male and Female?
Because pyreadstat.read_sav does not apply value formats unless you ask. apply_value_formats is False by default, so the dataframe holds the codes and to_csv writes exactly what it is given. Pass apply_value_formats=True and the labelled columns come back as text. This trips up people arriving from R, where foreign::read.spss applies labels unless told otherwise, so the same intent needs the opposite argument in each language.
Should I use pyreadstat or pandas.read_spss?
pandas.read_spss is a wrapper around pyreadstat, so it is not a way to avoid the dependency. Calling pyreadstat directly gives you the metadata object alongside the dataframe, which is where variable labels, value label mappings and missing value ranges live. Use read_spss when you want one line and none of that, and pyreadstat when you need to know what the file actually declared.
Do the defaults really differ between pandas and pyreadstat?
They do, and it catches people. pandas.read_spss converts categoricals unless you pass convert_categoricals=False, while pyreadstat.read_sav leaves values as codes unless you pass apply_value_formats=True. Two doors into the same library, opposite behaviour, and the CSV is where you find out which one you walked through.
Can Python read a .zsav file?
Yes. pyreadstat.read_sav reads both .sav and the compressed .zsav with the same call, so nothing needs converting first and a folder loop can glob both extensions.
How do I convert a .sav file that is bigger than my RAM?
pyreadstat.read_file_in_chunks wraps a reader and yields a dataframe at a time, so peak memory is one chunk rather than the whole file. Write the first chunk with a header and append the rest. There is also read_file_multiprocessing for spreading a whole file across processes, but that wants more memory rather than less, so on a file that already does not fit, chunking is the one you want.
Where did my 97, 98 and 99 codes go?
They were user-defined missing values, and pyreadstat collapses every one of them to NaN by default. Pass user_missing=True and they stay distinct, which matters when your questionnaire separates refused from does not apply from never asked. Once they are NaN there is no way to tell those three apart again.
My accented labels look wrong in Excel. Is the export broken?
Usually not. Excel on Windows opens a plain UTF-8 CSV as the system codepage, which turns accented characters into mojibake even though the file is correct. Write it with encoding="utf-8-sig" and Excel recognises the marker. If the text is already wrong in the dataframe before you write anything, that is a different problem: override what the file declared with the encoding argument on read, and cp1252 is the first thing worth trying.
How do I get the variable labels, not just the column names?
They are in the metadata, not the data. meta.column_names_to_labels maps each column to the question text it came from, and meta.variable_value_labels maps the coded variables to their answer labels. Neither travels in a CSV, so if you want them they need a second file. Tick the codebook option on a single file and the script writes one.
Related
The same conversion, a different language
- The same conversion in R with haven, foreign or rio, where the label default runs the other way.
Without writing any code at all
- Convert a .sav file here in the browser when you would rather not install anything at all.
- The compressed .zsav variant, which read_sav handles with the same call.
- SAV to JSON when the destination is an API rather than a spreadsheet.
- Open a .sav file and look at it first, which is often enough to see whether the labels are there at all.