Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Handling of times around daylight savings change #77

Open
tpetricek opened this issue Mar 30, 2014 · 4 comments
Open

Handling of times around daylight savings change #77

tpetricek opened this issue Mar 30, 2014 · 4 comments

Comments

@tpetricek
Copy link
Member

I suppose this will only reproduce when your computer is set to a timezone where the daylight savings time change happens on 30 March, but it should be easy to adapt:

(R.zoo([| 1.0 |], [| System.DateTime(2014,3,30).AddHours(0.0) |]) |> R.index).Print() 
// "2014-03-30 UTC"

(R.zoo([| 1.0 |], [| System.DateTime(2014,3,30).AddHours(1.0) |]) |> R.index).Print() 
// "2014-03-30 01:00:00 UTC"

(R.zoo([| 1.0 |], [| System.DateTime(2014,3,30).AddHours(2.0) |]) |> R.index).Print() 
// "2014-03-30 01:00:00 UTC"

(R.zoo([| 1.0 |], [| System.DateTime(2014,3,30).AddHours(3.0) |]) |> R.index).Print() 
// "2014-03-30 02:00:00 UTC"

The odd thing is that the second and third values print as the same time when they come back from R (possibly because at 1am, the UK clock jumps forward to 2am).

I have not investigated this further yet - so I'm not sure whether this is an issue with passing data into R or with getting data from R.

tpetricek added a commit to tpetricek/Deedle that referenced this issue Mar 30, 2014
Previously, these were implemented using floating window, which was terribly slow.
I added new basic operation to the index representing "shifting", so now we just
need to copy the data (skipping over the values that "drop out") and create a new
index (excluding the dropped out keys). I also added some documentation.

The performance improvement is orderds of magnitude (from seconds to milliseconds),
so I'm not attaching any specific numbers.

Aside, I also accidentally discovered fslaborg/RProvider#77
(and made the test here more stable) and I improved the build script to avoid
running tests twice.
@jmp75
Copy link
Contributor

jmp75 commented Mar 30, 2014

Is the issue fixed or fixable in the RProvider, or is this an issue at the level of R.NET that you need to work around ? I'd be interested to know; incidentally I do not remember any date-time specific support in R.NET yet. Given the ubiquitous nature of time series, it probably should. Dealt with it for http://rclr.codeplex.com, so I am very aware of the many quirks of the topic...

@tpetricek
Copy link
Member Author

There is some DateTime-specific code in the R provider here: https://github.com/BlueMountainCapital/FSharpRProvider/blob/master/src/RProvider/RInterop.fs#L123 So I suppose it might be R provider-specific? But I'm not sure..

@hmansell
Copy link
Contributor

I think this is just dodgy behavior in DateTime. In the Eastern US, where the clock springs forward an hour at 02:00 on March 9th, I get similar behavior.

> DateTime(2014,3,9).AddHours(1.0).ToUniversalTime();;
val it : DateTime = 2014-03-09T06:00:00
> DateTime(2014,3,9).AddHours(2.0).ToUniversalTime();;
val it : DateTime = 2014-03-09T07:00:00
> DateTime(2014,3,9).AddHours(3.0).ToUniversalTime();;
val it : DateTime = 2014-03-09T07:00:00

It appears that DateTime computation results in invalid local times:

> DateTime(2014,3,9).AddHours(2.1);;
val it : DateTime = 2014-03-09T02:06:00

Microsoft's advice is to basically never use DateTime to represent dates with times. They developed DateTimeOffset to overcome the majority of the issues.

@tpetricek
Copy link
Member Author

Yes, I suspect this might be the case. I'll try to change the tests and samples in Deedle to use DateTimeOffset.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants