Skip to content

Commit

Permalink
Add support for long integers
Browse files Browse the repository at this point in the history
Summary:
Here we improve the following:
 * The test case for long integers (big integers)
 * We have to use an unusual workaround to detect integer overflow of the `parse::<isize>` function as Meta is on a version of rust < 2022.
 * The workaround is documented and there is a todo as follows:
```
// TODO: use ParseIntError.kind() to detect integer overflow of
// parse of const_value when Meta is on rust 2022.
// In rust 2021 ParseIntError.kind is private
// For now, store an overflow Err from parsing a large integer
// Adapted from rust-lang/rust#22639
// and uutils/coreutils#2882
```

This would seem to be the recommended workaround for detecting integer overflow.

Reviewed By: stroxler

Differential Revision:
D42407915

Privacy Context Container: L1152058

fbshipit-source-id: 049dd6c74305fe193131fd1ea2ad72a0965d253a
  • Loading branch information
jasontatton authored and facebook-github-bot committed Jan 10, 2023
1 parent da445b3 commit c4973de
Show file tree
Hide file tree
Showing 3 changed files with 17 additions and 2 deletions.
15 changes: 15 additions & 0 deletions src/cst_to_ast.rs
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@
// https://github.com/tree-sitter/tree-sitter-python/blob/master/grammar.js

use std::collections::HashMap;
use std::num::ParseIntError;

use ast::Alias;
use ast::Arg;
Expand Down Expand Up @@ -74,6 +75,7 @@ pub struct Parser {
// contingent on if we are on lhs or rhs of assignment or del expression
current_expr_ctx: Vec<Option<ExprContext>>,
increment_expression_column_offset: isize,
integer_overflow_error: ParseIntError,
}

///
Expand Down Expand Up @@ -229,6 +231,7 @@ impl Parser {
ast_and_metadata: ASTAndMetaData::new(),
current_expr_ctx: Vec::new(),
increment_expression_column_offset: 0,
integer_overflow_error: "184467440737095516150".parse::<isize>().err().unwrap(),
}
}

Expand Down Expand Up @@ -3436,6 +3439,18 @@ impl Parser {

let integer_value = match const_value.parse::<isize>() {
Ok(value) => value,
Err(ref e) if *e == self.integer_overflow_error => {
// TODO: use ParseIntError.kind() to detect integer overflow of
// parse of const_value when Meta is on rust 2022.
// In rust 2021 ParseIntError.kind is private
// For now, store an overflow Err from parsing a large integer
// Adapted from https://github.com/rust-lang/rust/issues/22639
// and https://github.com/uutils/coreutils/pull/2882/
return Ok(ExprDesc::Constant {
value: Some(ConstantDesc::Num(Num::BigInt(const_value))),
kind: None,
});
}
Err(error_msg) => {
return Err(self.record_recoverable_error(
RecoverableError::UnexpectedExpression(format!(
Expand Down
3 changes: 2 additions & 1 deletion tests/test_resources/unit_tests/ast_literals.pytest
Original file line number Diff line number Diff line change
Expand Up @@ -231,7 +231,8 @@ f"{{{55}}}"
f"{{{{70 + 4}}}}"

## long integers
timestamp = 9223372036854775810
timestamp1 = 9223372036854775810
timestamp2 = -9223372036854775810

## complex numbers
complex_half = 0.0302988j
Expand Down

This file was deleted.

0 comments on commit c4973de

Please sign in to comment.