KeyError: 'vertical_type'

[damnjuhl]

At the end of going through all my friends (3k), no file is generated and this is the output. I attempted both storing the file to drive and to local with no success. Any ideas?

Traceback (most recent call last):
  File "src/fb2cal.py", line 814, in <module>
  File "src/fb2cal.py", line 117, in main
  File "src/fb2cal.py", line 401, in get_async_birthdays
  File "src/fb2cal.py", line 460, in parse_birthday_async_output
  File "src/fb2cal.py", line 699, in get_entity_id_from_vanity_name
KeyError: 'vertical_type'

I think it may be because of too many requests. Is there a way to split up and do smaller batches perhaps?

[mobeigi]

Yes almost definitely its because of the composer_query endpoint is enforcing rate limiting.

Excerpt from my blog:

Rate limiting note: If the composer query endpoint is hit enough in a short period of times, it seems to somehow restrict the number of entries returned. It seems to limit the query results to only returning results matching friends names, page names exactly. This limitation then disappears over time. So one should be careful how often or quickly they hit this endpoint!

Not sure how to solve this problem in a nice way. You could simply add a sleep between each call and hopefully it wouldn't trigger the rate limiting but I'm just not sure.

[damnjuhl]

Should I add the sleep timer after each of the get_ sections (line 401, line 699 from my first post) or is there a more ideal spot? [I'm a novice, apologies]

[mobeigi]

Note that once you're rate limited I found it can take a long time to get unrate limited (at times maybe 1-2 days).

Instead of adding a sleep which still might fail I suggest just commenting out the method 1 from this line:
https://github.com/mobeigi/fb2cal/blob/master/src/fb2cal.py#L694

From the method 1 comment until the logger.debug(composer_query_entries) line.

This will make it always use the slower fallback # Method 2: Scrape users profile page for entity id (significantly slower). Note this will be really slow so maybe leave the script for an extended period of time to complete (maybe overnight I'm not sure how slow it is). However, you should not get rate limited.

[damnjuhl]

That worked! I was still timed out ( a few hours after it finished ), but all the dates were pulled successfully, THANK YOU!

2700 contacts, with just method 2, took roughly 30 minutes. I was surprised at its snappiness.

[mobeigi]

Glad it worked!

Closing this as essentially a dupe of: #18