BLOG: How scikit-lego became dataframe-agnostic using Narwhals

[MarcoGorelli]

Text styling

• The blog is written with plain language (where relevant).
• If there are headers, they use the proper header tags in order to do so (with only one level-one header).
• All links describe where they link to (for example, check the Quansight labs website).
• Any kind of styling that the author uses (for example, bold for emphasis) is consistent throughout the blog.

Non-text contents

• Blog post featured image is in PNG or JPEG format, not SVG.
• All content is represented as text (for example, images need alt text and videos need captions or descriptive transcripts).
• If there are emojis, there are not more than three in a row.
• Don't use flashing gifs or videos.
• If it were to be read as plain text, the blog still makes sense and no information is missing.

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
labs	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Jun 17, 2024 6:53pm

[rec]

Very impressive results there, in a nice short article!

Comments are a couple of spelling errors, a quibble, and one tentative sentence change.

Good stuff!!!

[MarcoGorelli]

thanks @rec for your review, appreciate it! do you know why the CI job is failing? I've looked at the logs but they're pretty opaque to me

[rec]

To be honest, I'm a little shocked that adding a .md file would change anything in code execution.

Perhaps it has to do with the changes to .vscode/settings.json, which seem out-of-place here?

[rec]

Well, that suggestion of mine didn't work. :-/

This message: Warning: data for page "/blog/[slug]" (path "/blog/numpy-string-ufuncs") is 252 kB which exceeds the threshold of 128 kB, this amount of data can reduce performance. is baffling because you don't even mention that indirectly as far as I can see. (It's a warning but that triggers an error.)

I am unable to log in to vercel with my github account: could it be that a previous commit caused this error and you're blameless?

Likely you need to find someone who knows that system....

[koaning]

Suggested change

	Let's take a look at `sklego.pandas_utils.add_lags`. The code before version 0.9.0 did something like this:
	Let's take a look at `sklego.pandas_utils.add_lags` as a tangible example that demonstrates how you might be able to leverage narwhals in your own code. The code before version 0.9.0 did something like this:

[koaning]

This also reminds me, we probably want to rename that section of the API.

[koaning]

Suggested change

	to pandas.
	to pandas. Effectively, that would remove all the benefit of using a `LazyFrame` in the first place!

[MarcoGorelli]

not totally sure about this one - I mean:

df = pl.scan_parquet(...).with_columns(...).filter(...).group_by(...).agg(...).collect().to_pandas()
scikit_lego_func(df)

would still be better than

df = pl.read_parquet(...).with_columns(...).filter(...).group_by(...).agg(...).to_pandas()
scikit_lego_func(df)

[koaning]

Looks pretty good to me, left some comments to consider tho.

[MarcoGorelli]

Thanks all for your comments! 🙏

I think I've addressed everything

Any further comments, or do we want to ship this?

[rgommers]

This is a nice read. Thanks Marco for the post, and all reviewers for the review effort!

Any further comments, or do we want to ship this?

Looks like we're all good here, so I'll go ahead and hit the green button:)

[rgommers]

Link to the post for reference: https://labs.quansight.org/blog/scikit-lego-narwhals