Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bug: Flattening generic type also flattens all generic parameters #233

Merged
merged 24 commits into from
Feb 22, 2024

Conversation

NyxCode
Copy link
Collaborator

@NyxCode NyxCode commented Feb 12, 2024

Hey @escritorio-gustavo!
While trying to understand #232, I encountered this issue here. I'm not yet sure if it's the underlying cause, but maybe we should start with this.

Flattening an generic enum currently seems to not only flatten the enum, but also everything inside.
I've added a test to highlight this behaviour.

Any ideas what's going on here?

@NyxCode
Copy link
Collaborator Author

NyxCode commented Feb 12, 2024

Same thing happens if MyEnum is a generic struct instead.

@NyxCode NyxCode changed the title Bug: Flattening generic enum Bug: Flattening generic type also flattens all generic parameters Feb 12, 2024
@escritorio-gustavo
Copy link
Contributor

return quote!(
match <#generic_ident>::inline().as_str() {
// When exporting a generic, the default type used is `()`,
// which gives "null" when calling `.name()`. In this case, we
// want to preserve the type param's identifier as the name used
"null" => #generic_ident_str.to_owned(),
// If name is not "null", a type has been provided, so we use its
// name instead
x => x.to_owned()
}
);

Seems to be the issue. Changing inline to name fixes the new test, but causes the inline test in the same file to fail

@escritorio-gustavo
Copy link
Contributor

Every other test seems to be fine though

@NyxCode
Copy link
Collaborator Author

NyxCode commented Feb 12, 2024

Ah, I see, thanks!
I managed to hack smth together that works, but I get the feeling that our current test suite cannot be fully trusted.
I'll look into getting some fuzzing working, so that we can get a complete picture of what currently works and what doesn't.

@escritorio-gustavo
Copy link
Contributor

escritorio-gustavo commented Feb 12, 2024

This is a very annoying bug. It's enterely caused by what I did in #215 because Vec<T> wouldn't export properly when flattened

@gustavo-shigueo
Copy link
Collaborator

Hey @NyxCode, I'm at home right now, so I don't have access to my work account (@escritorio-gustavo).
I was trying something out and this seems to pass all the tests

return quote!(
    match <#generic_ident>::name().as_str() {
        "null" => #generic_ident_str.to_owned(),

        x if !std::any::type_name::<#generic_ident>().contains('<') => {
            x.to_owned()
        }

        _ => <#generic_ident>::inline()
    }
);

I am not sure if it is a good idea or if it breaks other stuff though

@escritorio-gustavo
Copy link
Contributor

Hey @NyxCode, I'm at home right now, so I don't have access to my work account (@escritorio-gustavo).

Just confirming, this was actually me

@escritorio-gustavo
Copy link
Contributor

If the macro could somehow read the value of type_name, convert the string into the type path and call name_with_type_args on it, that'd be somewhat better, but sadly, I don't think this is possible

@escritorio-gustavo
Copy link
Contributor

return quote!(
    match <#generic_ident>::name().as_str() {
        "null" => #generic_ident_str.to_owned(),

        x if !std::any::type_name::<#generic_ident>().contains('<') => {
            x.to_owned()
        }

        _ => <#generic_ident>::inline()
    }
);

I am not sure if it is a good idea or if it breaks other stuff though

This somewhat improves the situation, but still generates more inlining than expected. That is:

  #[derive(TS)]
  struct OtherType<T>(T);

  #[derive(TS)]
  struct SomeType<T>(OtherType<T>);

  #[derive(TS)]
  enum MyEnum<A, B> {
      VariantA(A),
      VariantB(B)
  }

  #[derive(TS)]
  struct Parent {
      e: MyEnum<i32, i32>,
      #[ts(inline)]
      e1: MyEnum<i32, SomeType<String>>
  }

  // This fails!
  // The #[ts(inline)] seems to inline recursively, so not only the definition of `MyEnum`, but
  // also the definition of `SomeType`.
  assert_eq!(
      Parent::decl(),
      "type Parent = { \
          e: MyEnum<number, number>, \
          e1: { \"VariantA\": number } | { \"VariantB\": SomeType<string> }, \
      }"
  );

Will fail, giving OtherType<string> instead of SomeType

@escritorio-gustavo
Copy link
Contributor

escritorio-gustavo commented Feb 14, 2024

I think it is no longer recursively inlining, but instead, it inlines one level deeper than it should. I'm really not sure though, especially because, the failure I described is the same as current behavior, even though the new test somehow passes

@escritorio-gustavo
Copy link
Contributor

escritorio-gustavo commented Feb 14, 2024

@NyxCode I think I've got something that might work, by changing T::name() so it now outputs Foo<T::name(), U::name(), ...>

I'll push the commit to a new branch and open a PR so you can inspect it. Please check out #235

@NyxCode
Copy link
Collaborator Author

NyxCode commented Feb 15, 2024

import type { LibraryType2<LibraryType2<ConsumerType>> } from "./LibraryType2";
import type { LibraryType2<LibraryType2<LibraryType1>> } from "./LibraryType2";

whoops 😆

@NyxCode
Copy link
Collaborator Author

NyxCode commented Feb 15, 2024

Not only are the import names broken, but our original logic of de-duping them doesn't work anymore (did it ever work with generics?).

ts-rs/ts-rs/src/export.rs

Lines 191 to 195 in 5daeed2

let deduplicated_deps = deps
.iter()
.filter(|dep| dep.type_id != TypeId::of::<T>())
.map(|dep| (&dep.ts_name, dep))
.collect::<BTreeMap<_, _>>();

@NyxCode
Copy link
Collaborator Author

NyxCode commented Feb 15, 2024

So there's something left to fix (besides adding tests) after all.
Anyway, @escritorio-gustavo, I pushed the changes here. Sorry for the one huge commit, though I added comments where I thought they might help. Let me know if anything is unclear.

@gustavo-shigueo
Copy link
Collaborator

Don't worry about it! I've already left work for today, so I'll take a good look at it tomorrow

@gustavo-shigueo
Copy link
Collaborator

Not only are the import names broken

I remember implementing some kind of hack to try and deal with that in #235, I'll check what I was and make it less hacky

@NyxCode
Copy link
Collaborator Author

NyxCode commented Feb 15, 2024

Not only are the import names broken

I remember implementing some kind of hack to try and deal with that in #235, I'll check what I was and make it less hacky

I've seen that! Seemed like a reasonable solution, though we could also add a const IDENT: &'static str or ident() -> String to TS to return the name without generics.
However, we're still using the TypeId for de-duping, and that changes if different generics are used.

@NyxCode
Copy link
Collaborator Author

NyxCode commented Feb 15, 2024

Ohh, I'm blind! I thought that was a BTreeSet, but we're collecting into a map, so we're just de-duping the names. Okay, easy fix then!

Comment on lines +291 to +306
///
/// If this type is generic, then all provided generic parameters will be swapped for
/// placeholders, resulting in a generic typescript definition.
/// Both `SomeType::<i32>::decl()` and `SomeType::<String>::decl()` will therefore result in
/// the same TypeScript declaration `type SomeType<A> = ...`.
fn decl() -> String {
panic!("{} cannot be declared", Self::name());
}

/// Declaration of this type using the supplied generic arguments.
/// The resulting TypeScript definition will not be generic. For that, see `TS::decl()`.
/// If this type is not generic, then this function is equivalent to `TS::decl()`.
fn decl_concrete() -> String {
panic!("{} cannot be declared", Self::name());
}

Copy link
Collaborator Author

@NyxCode NyxCode Feb 16, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here's the user-visible effect of this change.
For

struct Generic<A, B> { a: A, b: B }

Generic::<i32, String>::decl_concrete() returns type Generic = { a: i32, b: String };.
Generic::<i32, String>::decl() swaps out the generic parameters i32 and String out for on-the-fly generated dummy types A and B, resulting in type Generic<A, B> = { a: A, b: B };

Comment on lines 118 to 153
/// Generate a dummy unit struct for every generic type parameter of this type.
/// Example:
/// ```ignore
/// struct Generic<A, B, const C: usize> { /* ... */ }
/// ```
/// has two generic type parameters, `A` and `B`. This function will therefor generate
/// ```ignore
/// struct A;
/// impl ts_rs::TS for A { /* .. */ }
///
/// struct B;
/// impl ts_rs::TS for B { /* .. */ }
/// ```
fn generate_generic_types(&self) -> TokenStream {
let generics = self.generics.params.iter().filter_map(|g| match g {
GenericParam::Lifetime(_) => None,
GenericParam::Type(t) => Some(t.ident.clone()),
GenericParam::Const(_) => None,
});

quote! {
#(
#[derive(Copy, Clone, Debug, Hash, Eq, PartialEq, Ord, PartialOrd)]
struct #generics;
impl std::fmt::Display for #generics {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
write!(f, "{:?}", self)
}
}
impl TS for #generics {
fn name() -> String { stringify!(#generics).to_owned() }
fn transparent() -> bool { false }
}
)*
}
}
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here, these dummy types are generated, named the same as the generic parameters.
I thought about only defining one dummy type, so e.g #[docs(hidden)] pub struct Dummy<const NAME: &'static str>.
However, const-generics don't work with strings yet. Maybe something for the future to improve compile times.

Comment on lines 203 to 233
/// Generates the `decl()` and `decl_concrete()` methods.
/// `decl_concrete()` is simple, and simply defers to `inline()`.
/// For `decl()`, however, we need to change out the generic parameters of the type, replacing
/// them with the dummy types generated by `generate_generic_types()`.
fn generate_decl_fn(&mut self, rust_ty: &Ident) -> TokenStream {
let name = &self.ts_name;
let generic_types = self.generate_generic_types();
let ts_generics = format_generics(&mut self.dependencies, &self.generics);
// These are the generic parameters we'll be using.
let generic_idents = self.generics.params.iter().filter_map(|p| match p {
GenericParam::Lifetime(_) => None,
// Since we named our dummy types the same as the generic parameters, we can just keep
// the identifier of the generic parameter - its name is shadowed by the dummy struct.
GenericParam::Type(TypeParam { ident, .. }) => Some(quote!(#ident)),
// We keep const parameters as they are, since there's no sensible default value we can
// use instead. This might be something to change in the future.
GenericParam::Const(ConstParam { ident, .. }) => Some(quote!(#ident)),
});
quote! {
fn decl_concrete() -> String {
format!("type {} = {};", #name, Self::inline())
}
fn decl() -> String {
#generic_types
let inline = <#rust_ty<#(#generic_idents,)*> as ts_rs::TS>::inline();
let generics = #ts_generics;
format!("type {}{generics} = {inline};", #name)
}
}
}
}
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And finally, decl and decl_concrete are implemented here.
decl first creates the dummy types, shadowing the names of the generics (#generic_types). Then, I just call inline on that type.

Comment on lines -127 to +126
format_type(&syn::parse_str::<Type>(&type_as)?, dependencies, generics)
let ty = syn::parse_str::<Type>(&type_as)?;
quote!(<#ty as ts_rs::TS>::name())
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is just one example of how every use of generics::format_type got replaced with TS::name.

ts-rs/src/lib.rs Outdated Show resolved Hide resolved
@escritorio-gustavo
Copy link
Contributor

Hey @NyxCode! Is there anything else you'd like to change with this PR before merging?

@NyxCode
Copy link
Collaborator Author

NyxCode commented Feb 21, 2024

Hey @escritorio-gustavo !
I'm not sure! I think code-wise, it's fine with your changes (thanks!).
Ideally, I'd like some more tests to make sure everything still kinda works, especially the dependencies. But maybe that's something we can do after merging this, but before a new release.

@NyxCode NyxCode marked this pull request as ready for review February 21, 2024 18:54
@escritorio-gustavo
Copy link
Contributor

escritorio-gustavo commented Feb 21, 2024

I'd like some more tests to make sure everything still kinda works, especially the dependencies. But maybe that's something we can do after merging this, but before a new release.

Agreed. I'm thinking we could have a separate PR that

  • adds TS_DIR=tests-out to our test job
  • adds #[ts(export)] to pretty much every type created in our tests
  • runs tsc --no-emit on tests-out

This should hopefully increase our certainty about dependencies (I'm pretty sure #168 still needs to be solved though)

Ideally, I'd like some more tests to make sure everything still kinda works

Besides dependencies, what else do you think needs more thorough testing?

@escritorio-gustavo
Copy link
Contributor

Hmm... I think I'll have to unlink #232. Although this PR removes the unreachable! problem when not using inline, it doen't fix the second issue reported there (which, admitedly, is a duplicate of #168)

@escritorio-gustavo
Copy link
Contributor

It seems like inline only breaks dependencies for Result, Vec, HashMap and other handwritten impls where transparent returns true. We could make these just return Self::name or panic saying they can't be inlined

This idea seems ridiculous, but think about it:
Vec<T>::name returns Array<T::name()>
Vec<T>::inline returns Array<T::inline()>

But notice that Vec<T>::inline doesn't actually inline Vec, instead it inlines T.
Why? Because there is no way to inline Array.

The problem is instead of stating this to be illegal (or just the same as calling name), we recursively
inline T

@escritorio-gustavo
Copy link
Contributor

The same is true for Option and Result. name returns them already inlined! With any other enum, inlining gives the type union representation, while these two do that for name. There is no inlining an enum's type union form!

In fact, if Option were exported, name would return Option<T::name()> and inline would return what name currently returns

@escritorio-gustavo
Copy link
Contributor

escritorio-gustavo commented Feb 21, 2024

TL;DR

I think we should change the impls in ts-rs/lib.rs because:

  • Vec<T>::inline, HashMap<K, V>::inline, [T; N]::inline can't exist, since there is no inlining for Array<T>, Record<K, V> or [T, T, ...], so inline should just defer to name
  • Option<T>::name and Result<T, E>::name can't exist either, since Result and Option aren't exported so name should defer to inline

@escritorio-gustavo
Copy link
Contributor

escritorio-gustavo commented Feb 21, 2024

Alternatively, to keep current behavior, we have to recursively search for a type that isn't of ot those I mentioned (or their impl_shadows/impl_wrappers) to do dependencies.append_from

@escritorio-gustavo
Copy link
Contributor

@NyxCode I'll merge this PR and open a discussion for us to try and figure out how to best deal with all of these other issues

@escritorio-gustavo escritorio-gustavo merged commit 4c3349c into main Feb 22, 2024
8 checks passed
@escritorio-gustavo escritorio-gustavo deleted the bug/flatten-generic-enum branch February 22, 2024 12:30
@escritorio-gustavo
Copy link
Contributor

The discussion link: #239

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Incorrect TS types generated from types containing type aliases
3 participants