openhatch

Issue773

Title Github importer is generating ParsedBug objects with no _tracker_name value
Milestone Priority bug
Waiting On Status resolved
Superseder Nosy List jwm, paulproteus
Assigned To Keywords

Created on 2012-09-26.05:15:42 by paulproteus, last changed 2012-09-27.01:20:46 by paulproteus.

Messages
msg3436 (view) Author: paulproteus Date: 2012-09-27.01:20:46
This is in. Could use more test coverage, but it seems to pass what tests there 
are. (-:
msg3435 (view) Author: jwm Date: 2012-09-27.01:01:19
https://github.com/openhatch/oh-bugimporters/pull/21
msg3431 (view) Author: paulproteus Date: 2012-09-26.22:22:30
It seems to (at least) be true of the Github importer.
msg3430 (view) Author: paulproteus Date: 2012-09-26.05:15:41
I am running "python import_bugimporter_data /tmp/results.jsonlines".

What I expect is that it should import JSON data from the file specified without 
printing any warnings.

What I'm seeing instead is this error:

  File "/home/deploy/milestone-a/mysite/customs/management/commands/import_bugi
mporter_data.py", line 49, in handle
    mysite.customs.core_bugimporters.import_one_bug_item(bug_dict)
  File "/home/deploy/milestone-a/mysite/customs/core_bugimporters.py", line 79,
 in import_one_bug_item
    tracker_name=d['_tracker_name'])
KeyError: '_tracker_name'

I believe we should adjust oh-bugimporters so that it refuses to export any 
ParsedBug item that doesn't have a _tracker_name value.

(My guess for the root cause here is that there's probably a bug in one of the 
bugimporter classes where it's not filling in the _tracker_name value.)

We can possibly do this with the help of an undocumented property of the scrapy 
Field class: "required". You can see that here: 
http://pydoc.net/s01.scrapy/0.12.3/s01.scrapy.fieldproperty

An alternative is to add some of our own machinery that insists ParsedBug has 
this field set to a value.
History
Date User Action Args
2012-09-27 01:20:46paulproteussetstatus: chatting -> resolved
messages: + msg3436
2012-09-27 01:01:19jwmsetnosy: + jwm
messages: + msg3435
2012-09-26 22:22:31paulproteussetstatus: unread -> chatting
messages: + msg3431
title: Some oh-bugimporters importer is generating ParsedBug objects with no _tracker_name value -> Github importer is generating ParsedBug objects with no _tracker_name value
2012-09-26 05:15:42paulproteuscreate