Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Default format for date on mapping property do not match default format used to save when calling save all #11

Open
ederfmartins opened this issue Jan 7, 2016 · 1 comment
Assignees
Labels

Comments

@ederfmartins
Copy link
Contributor

elasticsearch version: 2.1
python version: 2.7
elasticsearch python version: 2.1.0

In esengine/fields.py: Default format for date on mapping property do not match default format used to save when calling save_all (that uses value.isoformat() internally).

from esengine.fields import DateField
from esengine import Document
from elasticsearch import Elasticsearch


class MyDoc(Document):
   _doctype = 'mydoctype'
   _index = 'myindex'
   _es = Elasticsearch()
   mydate = DateField()

MyDoc.put_mapping()

try:
   MyDoc(mydate='2015-10-01').save()
except Exception as e:
   print e

try:
   MyDoc.save_all([MyDoc(mydate='2015-10-01')])
except Exception as e:
   print e

traceback

Traceback (most recent call last):
 File "t.py", line 13, in <module>
   MyDoc.save_all([MyDoc(mydate='2015-10-01')])
 File "/usr/local/lib/python2.7/dist-packages/esengine/document.py", line 384, in save_all
   eh.bulk(cls.get_es(es), actions, **kwargs)
 File "/usr/local/lib/python2.7/dist-packages/elasticsearch/helpers/__init__.py", line 188, in bulk
   for ok, item in streaming_bulk(client, actions, **kwargs):
 File "/usr/local/lib/python2.7/dist-packages/elasticsearch/helpers/__init__.py", line 160, in streaming_bulk
   for result in _process_bulk_chunk(client, bulk_actions, raise_on_exception, raise_on_error, **kwargs):
 File "/usr/local/lib/python2.7/dist-packages/elasticsearch/helpers/__init__.py", line 132, in _process_bulk_chunk
   raise BulkIndexError('%i document(s) failed to index.' % len(errors), errors)
elasticsearch.helpers.BulkIndexError: (u'1 document(s) failed to index.', [{u'create': {u'status': 400, u'_type': u'mydoctype', u'_id': u'AVIcSiT2s5zkWhhVSfUA', u'error': {u'caused_by': {u'reason': u'Invalid format: "2015-10-01T00:00:00" is malformed at "T00:00:00"', u'type': u'illegal_argument_exception'}, u'reason': u'failed to parse [mydate]', u'type': u'mapper_parsing_exception'}, u'_index': u'myindex'}}])

probably removing "format": "yyyy-MM-dd HH:mm:ss||yyyy-MM-dd" solves the problem

class DateField(BaseField):
...
    _default_mapping = {
        "type": "date",
        "format": "yyyy-MM-dd HH:mm:ss||yyyy-MM-dd"
    }
...
    def to_dict(self, value, validate=True):
        if validate:
            self.validate(value)
        if value:
            if self._date_format:
                return value.strftime(self._date_format)
            else:
                return value.isoformat()
@ederfmartins ederfmartins self-assigned this Jan 7, 2016
@rochacbruno
Copy link
Contributor

"format": "yyyy-MM-dd HH:mm:ss||yyyy-MM-dd" removed, but still needs to write a test to run on Travis.

if os.environ.get('TRAVIS'):
    assert.....

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants