LMT entries missing for some time zones

[ThePasko]

Reposting from moment/moment#3001

When converting a specific datetime value to GMT or UTC the output is a bit confusing:

moment.tz("1751-01-01T00:00:00", "Europe/Dublin").clone().tz("Europe/London").format("YYYY-MM-DDTHH:mm:ss.SSSS") -> "1751-01-01T00:25:21.0000"

or this

moment.tz("1751-01-01T00:00:00", "Europe/Dublin").clone().utc().format("YYYY-MM-DDTHH:mm:ss.SSSS") -> "1751-01-01T00:25:21.0000"

The first thing I learned here is that the 25min added is actually not a bug :)
But what about the extra 21 seconds added in both cases?? Is that a bug?

[maggiepint]

My notes from the issue in the moment repo:

It appears that the Europe/Dublin timezone did indeed have a -00:25:21 offset from August 2nd 1880 to October 1st 1916.
Search for Europe/Dublin in the following file:
ftp://ftp.iana.org/tz/data/europe

I realize that your time range predates this. I think there may be something going on with how far back the IANA data goes in Moment-timezone.

[mattjohnsonpint]

This is valid. You can review the entry from the tzdb for Europe/Dublin.

You can also read about it on Wikipedia.

[mattjohnsonpint]

Actually, I'm going to re-open this, but only because it appears that we are somehow not importing the LMT value. The DMT value should only be in effect from 1880 to 1916. For the 1751 date provided, it should be using the LMT value of -0:25:00.

Though do keep in mind that really anything before 1970 is suspicious. The TZDB doesn't guarantee historical accuracy. It does the best it can from sources reported, but human beings really didn't keep time all that accurately in the 18th century.

[mattjohnsonpint]

You can see the Europe/Dublin entry in moment-timezone here.

At first, I thought we were skipping LMT for all time zones, but I see many of them that have it, and many that do not. @timrwood ?

[timrwood]

I think this is due to limitations from zdump, which we use to generate data files.

zdump only outputs data between 1901-12-13 and 2038-01-19, which is the range of a 32 bit signed unix timestamp in seconds.

UTC  Fri Dec 13 20:45:52 1901 UTC = Fri Dec 13 20:45:52 1901 UTC isdst=0
UTC  Sat Dec 14 20:45:52 1901 UTC = Sat Dec 14 20:45:52 1901 UTC isdst=0
UTC  Mon Jan 18 03:14:07 2038 UTC = Mon Jan 18 03:14:07 2038 UTC isdst=0
UTC  Tue Jan 19 03:14:07 2038 UTC = Tue Jan 19 03:14:07 2038 UTC isdst=0

[mattjohnsonpint]

Hmmm.. Doing some research and I can't find that as a stated limitation. A few mentions related to 32-bit vs 64-bit.. Are we using 32 bit?

Also - how would you feel about replacing zdump with something more platform neutral? We could likely swap it out for pytz (python). Then it would build on Linux, Mac and Windows, and we'd have better control over the source data. (Currently you can't build moment-timezone on Windows)

Alternatively, we could look at writing our own tzdata parser in JavaScript.

[timrwood]

The source for zic.c and zdump.c are both available, and we could easily download them as part of the download task.

I would much prefer to compile those as part of the build process than introduce another language dependency.

[mattjohnsonpint]

Unfortunately, that wouldn't work. We can't count on a C compiler being available, and tzcode (including zic and zdump) doesn't build on standard Windows due to environment issues, regardless of compiler choice.

There were some recent changes posted to tzcode that are supposed to let it compile under MinGW, but I haven't tried it yet. Even then, MinGW is a very large overhead. I don't think the typical JS developer would go for that.

[mattjohnsonpint]

FYI - I just spent the last couple of hours trying to get zic and zdump to build on mingw with no luck. I've tried cygwin, msys, and win-builds, to no avail. I'll see if I can get help from the tz list, but I still think we should be considering either a python or pure javascript solution.

[mattjohnsonpint]

I finally have been able to make this work, getting zic and zdump to work on windows, as well as fixing various errors with Linux output of zic and zdump as well.

I'm not including zic.exe or zdump.exe in our builds, but instructions for building are in the tz list archives here.

Various grunt tasks needed updating to support this. These are in release 0.5.22, along with the repaired LMT entries.