进入scrapy相关输出csv的源码路径
我的路径为:D:\Python35\Lib\site-packages\scrapy
编辑该路径下的exporters.py文件,并修改的内容如下:
- class CsvItemExporter(BaseItemExporter):
- def __init__(self, file, include_headers_line=True, join_multivalued=‘,‘, **kwargs):
- self._configure(kwargs, dont_fail=True)
- if not self.encoding:
- self.encoding = ‘utf-8‘
- self.include_headers_line = include_headers_line
- self.stream = io.TextIOWrapper(
- file,
- newline=‘‘, #在这里添加newline=‘‘,
- line_buffering=False,
- write_through=True,
- encoding=self.encoding
- ) if six.PY3 else file
- self.csv_writer = csv.writer(self.stream, **kwargs)
- self._headers_not_written = True
- self._join_multivalued = join_multivalued
完成之后进行测试,成功之后下面设计scrapy的固定scv字段的输出:
首先在setting.py的同级目录下创建一个名为gl_csv_item_exporter.py的文件并写入:
- .conf settings
- .exporters CsvItemExporter
- (CsvItemExporter):
- (*args**kwargs):
- delimiter = settings.get()
- kwargs[] = delimiter
- fields_to_export = settings.get([])
- (MyProjectCsvItemExporter).(*args**kwargs)
在setting.py下添加以下内容:(itjz是我的项目名,使用的时候改一下)
注意:title2和title是我项目里定义的item名
- FEED_EXPORTERS = {
- ‘csv‘: ‘itjz.gl_csv_item_exporter.MyProjectCsvItemExporter‘,
- }
- FIELDS_TO_EXPORT = [
- ‘title2‘,
- ‘title‘
- ]
来源: http://www.bubuko.com/infodetail-2305316.html