13.1. csv — CSV 文件读写

2.3 新版功能.

The so-called CSV (Comma Separated Values) format is the most common import and export format for spreadsheets and databases. There is no “CSV standard”, so the format is operationally defined by the many applications which read and write it. The lack of a standard means that subtle differences often exist in the data produced and consumed by different applications. These differences can make it annoying to process CSV files from multiple sources. Still, while the delimiters and quoting characters vary, the overall format is similar enough that it is possible to write a single module which can efficiently manipulate such data, hiding the details of reading and writing the data from the programmer.

The csv module implements classes to read and write tabular data in CSV format. It allows programmers to say, “write this data in the format preferred by Excel,” or “read data from this file which was generated by Excel,” without knowing the precise details of the CSV format used by Excel. Programmers can also describe the CSV formats understood by other applications or define their own special-purpose CSV formats.

csv 模块中的 reader 类和 writer 类可用于读写序列化的数据。也可使用 DictReader 类和 DictWriter 类以字典的形式读写数据。

注解

This version of the csv module doesn’t support Unicode input. Also, there are currently some issues regarding ASCII NUL characters. Accordingly, all input should be UTF-8 or printable ASCII to be safe; see the examples in section 例子.

参见

该实现在“Python 增强提议” - PEP 305 (CSV 文件 API) 中被提出

《Python 增强提议》提出了对 Python 的这一补充。

13.1.1. 模块内容

csv 模块定义了以下函数：

csv.reader(csvfile, dialect='excel', **fmtparams)

Return a reader object which will iterate over lines in the given csvfile. csvfile can be any object which supports the iterator protocol and returns a string each time its next() method is called — file objects and list objects are both suitable. If csvfile is a file object, it must be opened with the ‘b’ flag on platforms where that makes a difference. An optional dialect parameter can be given which is used to define a set of parameters specific to a particular CSV dialect. It may be an instance of a subclass of the Dialect class or one of the strings returned by the list_dialects() function. The other optional fmtparams keyword arguments can be given to override individual formatting parameters in the current dialect. For full details about the dialect and formatting parameters, see section 变种与格式参数.

Each row read from the csv file is returned as a list of strings. No automatic data type conversion is performed.

一个简短的用法示例:

>>> import csv
>>> with open('eggs.csv', 'rb') as csvfile:
...     spamreader = csv.reader(csvfile, delimiter=' ', quotechar='|')
...     for row in spamreader:
...         print ', '.join(row)
Spam, Spam, Spam, Spam, Spam, Baked Beans
Spam, Lovely Spam, Wonderful Spam

在 2.5 版更改: The parser is now stricter with respect to multi-line quoted fields. Previously, if a line ended within a quoted field without a terminating newline character, a newline would be inserted into the returned field. This behavior caused problems when reading files which contained carriage return characters within fields. The behavior was changed to return the field without inserting newlines. As a consequence, if newlines embedded within fields are important, the input should be split into lines in a manner which preserves the newline characters.

csv.writer(csvfile, dialect='excel', **fmtparams)

Return a writer object responsible for converting the user’s data into delimited strings on the given file-like object. csvfile can be any object with a write() method. If csvfile is a file object, it must be opened with the ‘b’ flag on platforms where that makes a difference. An optional dialect parameter can be given which is used to define a set of parameters specific to a particular CSV dialect. It may be an instance of a subclass of the Dialect class or one of the strings returned by the list_dialects() function. The other optional fmtparams keyword arguments can be given to override individual formatting parameters in the current dialect. For full details about the dialect and formatting parameters, see section 变种与格式参数. To make it as easy as possible to interface with modules which implement the DB API, the value None is written as the empty string. While this isn’t a reversible transformation, it makes it easier to dump SQL NULL data values to CSV files without preprocessing the data returned from a cursor.fetch* call. Floats are stringified with repr() before being written. All other non-string data are stringified with str() before being written.

一个简短的用法示例:

import csv
with open('eggs.csv', 'wb') as csvfile:
    spamwriter = csv.writer(csvfile, delimiter=' ',
                            quotechar='|', quoting=csv.QUOTE_MINIMAL)
    spamwriter.writerow(['Spam'] * 5 + ['Baked Beans'])
    spamwriter.writerow(['Spam', 'Lovely Spam', 'Wonderful Spam'])

csv.register_dialect(name, [dialect, ]**fmtparams)

Associate dialect with name. name must be a string or Unicode object. The dialect can be specified either by passing a sub-class of Dialect, or by fmtparams keyword arguments, or both, with keyword arguments overriding parameters of the dialect. For full details about the dialect and formatting parameters, see section 变种与格式参数.

csv.unregister_dialect(name)

从变种注册表中删除 name 对应的变种。如果 name 不是已注册的变种名称，则抛出 Error 异常。

csv.get_dialect(name)

Return the dialect associated with name. An Error is raised if name is not a registered dialect name.

在 2.5 版更改: This function now returns an immutable Dialect. Previously an instance of the requested dialect was returned. Users could modify the underlying class, changing the behavior of active readers and writers.

csv.list_dialects()

返回所有已注册变种的名称。

csv.field_size_limit([new_limit])

返回解析器当前允许的最大字段大小。如果指定了 new_limit，则它将成为新的最大字段大小。

2.5 新版功能.

csv 模块定义了以下类：

class csv.DictReader(f, fieldnames=None, restkey=None, restval=None, dialect='excel', *args, **kwds)

Create an object which operates like a regular reader but maps the information read into a dict whose keys are given by the optional fieldnames parameter. The fieldnames parameter is a sequence whose elements are associated with the fields of the input data in order. These elements become the keys of the resulting dictionary. If the fieldnames parameter is omitted, the values in the first row of the file f will be used as the fieldnames. If the row read has more fields than the fieldnames sequence, the remaining data is added as a sequence keyed by the value of restkey. If the row read has fewer fields than the fieldnames sequence, the remaining keys take the value of the optional restval parameter. Any other optional or keyword arguments are passed to the underlying reader instance.

一个简短的用法示例:

>>> import csv
>>> with open('names.csv') as csvfile:
...     reader = csv.DictReader(csvfile)
...     for row in reader:
...         print(row['first_name'], row['last_name'])
...
Baked Beans
Lovely Spam
Wonderful Spam

class csv.DictWriter(f, fieldnames, restval='', extrasaction='raise', dialect='excel', *args, **kwds)

Create an object which operates like a regular writer but maps dictionaries onto output rows. The fieldnames parameter is a sequence of keys that identify the order in which values in the dictionary passed to the writerow() method are written to the file f. The optional restval parameter specifies the value to be written if the dictionary is missing a key in fieldnames. If the dictionary passed to the writerow() method contains a key not found in fieldnames, the optional extrasaction parameter indicates what action to take. If it is set to 'raise' a ValueError is raised. If it is set to 'ignore', extra values in the dictionary are ignored. Any other optional or keyword arguments are passed to the underlying writer instance.

Note that unlike the DictReader class, the fieldnames parameter of the DictWriter is not optional. Since Python’s dict objects are not ordered, there is not enough information available to deduce the order in which the row should be written to the file f.

一个简短的用法示例:

import csv

with open('names.csv', 'w') as csvfile:
    fieldnames = ['first_name', 'last_name']
    writer = csv.DictWriter(csvfile, fieldnames=fieldnames)

    writer.writeheader()
    writer.writerow({'first_name': 'Baked', 'last_name': 'Beans'})
    writer.writerow({'first_name': 'Lovely', 'last_name': 'Spam'})
    writer.writerow({'first_name': 'Wonderful', 'last_name': 'Spam'})

class csv.Dialect

Dialect 类是主要依赖于其属性的容器类，用于将定义好的参数传递给特定的 reader 或 writer 实例。

class csv.excel

excel 类定义了 Excel 生成的 CSV 文件的常规属性。它在变种注册表中的名称是 'excel'。

class csv.excel_tab

excel_tab 类定义了 Excel 生成的、制表符分隔的 CSV 文件的常规属性。它在变种注册表中的名称是 'excel-tab'。

class csv.Sniffer

Sniffer 类用于推断 CSV 文件的格式。

Sniffer 类提供了两个方法：

sniff(sample, delimiters=None)

分析给定的 sample 并返回一个 Dialect 子类，该子类中包含了分析出的格式参数。如果给出可选的 delimiters 参数，则该参数会被解释为字符串，该字符串包含了可能的有效分隔符。

has_header(sample)

分析示例文本（假定为 CSV 格式），如果第一行很可能是一系列列标题，则返回 True。

使用 Sniffer 的示例：

with open('example.csv', 'rb') as csvfile:
    dialect = csv.Sniffer().sniff(csvfile.read(1024))
    csvfile.seek(0)
    reader = csv.reader(csvfile, dialect)
    # ... process CSV file contents here ...

csv 模块定义了以下常量：

csv.QUOTE_ALL

指示 writer 对象给所有字段加上引号。

csv.QUOTE_MINIMAL

指示 writer 对象仅为包含特殊字符（例如 定界符、引号字符 或 行结束符 中的任何字符）的字段加上引号。

csv.QUOTE_NONNUMERIC

指示 writer 对象为所有非数字字段加上引号。

指示 reader 将所有未用引号引出的字段转换为 float 类型。

csv.QUOTE_NONE

指示 writer 对象不使用引号引出字段。当 定界符 出现在输出数据中时，其前面应该有 转义符。如果未设置 转义符，则遇到任何需要转义的字符时，writer 都会抛出 Error 异常。

指示 reader 不对引号字符进行特殊处理。

csv 模块定义了以下异常：

exception csv.Error

该异常可能由任何发生错误的函数抛出。

13.1.2. 变种与格式参数

为了更容易指定输入和输出记录的格式，特定的一组格式参数组合为一个 dialect（变种）。一个 dialect 是一个 Dialect 类的子类，它具有一组特定的方法和一个 validate() 方法。创建 reader 或 writer 对象时，程序员可以将某个字符串或 Dialect 类的子类指定为 dialect 参数。要想补充或覆盖 dialect 参数，程序员还可以单独指定某些格式参数，这些参数的名称与下面 Dialect 类定义的属性相同。

Dialect 类支持以下属性：

Dialect.delimiter

一个用于分隔字段的单字符，默认为 ','。

Dialect.doublequote

控制出现在字段中的 引号字符 本身应如何被引出。当该属性为 True 时，双写引号字符。如果该属性为 False，则在 引号字符 的前面放置 转义符。默认值为 True。

在输出时，如果 doublequote 是 False，且 转义符 未指定，且在字段中发现 引号字符 时，会抛出 Error 异常。

Dialect.escapechar

一个用于 writer 的单字符，用来在 quoting 设置为 QUOTE_NONE 的情况下转义 定界符，在 doublequote 设置为 False 的情况下转义 引号字符。在读取时，escapechar 去除了其后所跟字符的任何特殊含义。该属性默认为 None，表示禁用转义。

Dialect.lineterminator

放在 writer 产生的行的结尾，默认为 '\r\n'。

注解

reader 经过硬编码，会识别 '\r' 或 '\n' 作为行尾，并忽略 lineterminator。未来可能会更改这一行为。

Dialect.quotechar

一个单字符，用于包住含有特殊字符的字段，特殊字符如 定界符 或 引号字符 或换行符。默认为 '"'。

Dialect.quoting

控制 writer 何时生成引号，以及 reader 何时识别引号。该属性可以等于任何 QUOTE_* 常量（参见 模块内容 段落），默认为 QUOTE_MINIMAL。

Dialect.skipinitialspace

如果为 True，则忽略 定界符 之后的空格。默认值为 False。

Dialect.strict

如果为 True，则在输入错误的 CSV 时抛出 Error 异常。默认值为 False。

13.1.3. Reader 对象

Reader 对象（DictReader 实例和 reader() 函数返回的对象）具有以下公开方法：

csvreader.next()

Return the next row of the reader’s iterable object as a list, parsed according to the current dialect.

Reader 对象具有以下公开属性：

csvreader.dialect

变种描述，只读，供解析器使用。

csvreader.line_num

源迭代器已经读取了的行数。它与返回的记录数不同，因为记录可能跨越多行。

2.5 新版功能.

DictReader 对象具有以下公开属性：

csvreader.fieldnames

字段名称。如果在创建对象时未传入字段名称，则首次访问时或从文件中读取第一条记录时会初始化此属性。

在 2.6 版更改.

13.1.4. Writer 对象

Writer objects (DictWriter instances and objects returned by the writer() function) have the following public methods. A row must be a sequence of strings or numbers for Writer objects and a dictionary mapping fieldnames to strings or numbers (by passing them through str() first) for DictWriter objects. Note that complex numbers are written out surrounded by parens. This may cause some problems for other programs which read CSV files (assuming they support complex numbers at all).

csvwriter.writerow(row)

将 row 形参写入 writer 的文件对象，并按照当前设定形式进行格式化。

csvwriter.writerows(rows)

将 rows*（即能迭代出多个上述 *row 对象的迭代器）中的所有元素写入 writer 的文件对象，并根据当前设置的变种进行格式化。

Writer 对象具有以下公开属性：

csvwriter.dialect

变种描述，只读，供 writer 使用。

DictWriter 对象具有以下公开方法：

DictWriter.writeheader()

使用（构造器所规定的）字段名写入一行。

2.7 新版功能.

13.1.5. 例子

读取 CSV 文件最简单的一个例子:

import csv
with open('some.csv', 'rb') as f:
    reader = csv.reader(f)
    for row in reader:
        print row

读取其他格式的文件:

import csv
with open('passwd', 'rb') as f:
    reader = csv.reader(f, delimiter=':', quoting=csv.QUOTE_NONE)
    for row in reader:
        print row

相应最简单的写入示例是:

import csv
with open('some.csv', 'wb') as f:
    writer = csv.writer(f)
    writer.writerows(someiterable)

注册一个新的变种:

import csv
csv.register_dialect('unixpwd', delimiter=':', quoting=csv.QUOTE_NONE)
with open('passwd', 'rb') as f:
    reader = csv.reader(f, 'unixpwd')

Reader 的更高级用法——捕获并报告错误:

import csv, sys
filename = 'some.csv'
with open(filename, 'rb') as f:
    reader = csv.reader(f)
    try:
        for row in reader:
            print row
    except csv.Error as e:
        sys.exit('file %s, line %d: %s' % (filename, reader.line_num, e))

尽管该模块不直接支持解析字符串，但仍可如下轻松完成:

import csv
for row in csv.reader(['one,two,three']):
    print row

The csv module doesn’t directly support reading and writing Unicode, but it is 8-bit-clean save for some problems with ASCII NUL characters. So you can write functions or classes that handle the encoding and decoding for you as long as you avoid encodings like UTF-16 that use NULs. UTF-8 is recommended.

unicode_csv_reader() below is a generator that wraps csv.reader to handle Unicode CSV data (a list of Unicode strings). utf_8_encoder() is a generator that encodes the Unicode strings as UTF-8, one string (or row) at a time. The encoded strings are parsed by the CSV reader, and unicode_csv_reader() decodes the UTF-8-encoded cells back into Unicode:

import csv

def unicode_csv_reader(unicode_csv_data, dialect=csv.excel, **kwargs):
    # csv.py doesn't do Unicode; encode temporarily as UTF-8:
    csv_reader = csv.reader(utf_8_encoder(unicode_csv_data),
                            dialect=dialect, **kwargs)
    for row in csv_reader:
        # decode UTF-8 back to Unicode, cell by cell:
        yield [unicode(cell, 'utf-8') for cell in row]

def utf_8_encoder(unicode_csv_data):
    for line in unicode_csv_data:
        yield line.encode('utf-8')

For all other encodings the following UnicodeReader and UnicodeWriter classes can be used. They take an additional encoding parameter in their constructor and make sure that the data passes the real reader or writer encoded as UTF-8:

import csv, codecs, cStringIO

class UTF8Recoder:
    """
    Iterator that reads an encoded stream and reencodes the input to UTF-8
    """
    def __init__(self, f, encoding):
        self.reader = codecs.getreader(encoding)(f)

    def __iter__(self):
        return self

    def next(self):
        return self.reader.next().encode("utf-8")

class UnicodeReader:
    """
    A CSV reader which will iterate over lines in the CSV file "f",
    which is encoded in the given encoding.
    """

    def __init__(self, f, dialect=csv.excel, encoding="utf-8", **kwds):
        f = UTF8Recoder(f, encoding)
        self.reader = csv.reader(f, dialect=dialect, **kwds)

    def next(self):
        row = self.reader.next()
        return [unicode(s, "utf-8") for s in row]

    def __iter__(self):
        return self

class UnicodeWriter:
    """
    A CSV writer which will write rows to CSV file "f",
    which is encoded in the given encoding.
    """

    def __init__(self, f, dialect=csv.excel, encoding="utf-8", **kwds):
        # Redirect output to a queue
        self.queue = cStringIO.StringIO()
        self.writer = csv.writer(self.queue, dialect=dialect, **kwds)
        self.stream = f
        self.encoder = codecs.getincrementalencoder(encoding)()

    def writerow(self, row):
        self.writer.writerow([s.encode("utf-8") for s in row])
        # Fetch UTF-8 output from the queue ...
        data = self.queue.getvalue()
        data = data.decode("utf-8")
        # ... and reencode it into the target encoding
        data = self.encoder.encode(data)
        # write to the target stream
        self.stream.write(data)
        # empty queue
        self.queue.truncate(0)

    def writerows(self, rows):
        for row in rows:
            self.writerow(row)