Early access ⭕ Google GitHub / Python agent

[xa4hf8]

An experimental A.I. agent that automates Python and JavaScript coding tasks within GitHub workflows. (Google account required.)

http://labs.google.com/jules

[markqvist]

For a while now, I've actually been thinking about adding a section to the contribution guidelines very specifically prohibiting all contributions that contains even the bare trace of any sort of ML/generative code.

Guess it's time to get that one checked off the list.

So just to be clear: Absolutely fucking not.

[faragher]

Briefly, you swung by and posted a "you should use this pre-release Google technology" and posted a screamsheet image. If you're not a paid "viral marketer" you need to work on your presentation skills.

Second, you're talking to someone trained in . . . let's call it persuasive writing. This stuff doesn't work on me. You found two P.R.s and didn't bother to discuss their content, simply attempted to use them to make it feel like only your product can save us from the terrible fate which is ourselves. Oh, no, we lack the resources to manage a project effectively, help us random person on the Internet!

Then an article about people using ChatGPT? About how doctors trust their training over a LLM that simply collates all the data it finds into a chatbot (thus Chat GPT)? The same chatbots that told me Reticulum was written in Rust? And when I asked where it found that information, it said "Whoops, I don't know where I saw that, I guess I'm wrong."

So, your arguments and the way you present them are quite nonsensical. I assume you picked up a job on Mechanical Turk or whatever Google is using these days. No harm, you've gotta eat, but I work with Mark specifically because I trust his judgment on things like this. Allow me to explain why this is a non-starter for Reticulum.

It cannot produce workable code for the project.

Since many of the classes and functions in Reticulum are unique, there are no sources for LLMs to draw on except Reticulum code. Even if it generated functional code, it would have to be sifted through line by line, its logic reverse engineered and monitored to make sure not only that it actually did what was required, but that it didn't introduce unintended issues. This is precisely the reason human-made mass refactoring is rejected. The time spent going over each and every line to ensure intended behavior is greater than a human simply making the code themselves.

Similarly, a well enough defined specification from which AI could generate functional code is effectively the code itself. There's little to no benefit in using a system such as this.

It cannot provide effective tests.

Testing code is just as important as the codebase itself. You need to find deterministic and logical mechanisms to test if code works. These mechanisms need to be implemented in a procedural way to ensure both scientific rigor, but to not miss any edge cases. The simplest examples are how random cases can miss a flaw simply because that number is never called, effective systems that are misused will miss underlying assumptions (a car assumes its hull is not taking on water), and how human intervention can bypass what appear to be solid tests (see MHRD, a game where people routinely pass off invalid solutions because they build to the test, not the functionality).

Since every test needs to be repeatable across every situation, past, present, and future, this means the test needs to be absolutely rigorous based off of not a list of outputs but an understanding of underlying principles, and AI cannot do this. Take the classic F=MA. It's used very widely, and would likely make a functional test; it's wrong. It's actually F=γMA, where the Lorentz factor (γ) is the inverse square root of 1 - v^2 / c^c . A test on virtually any terrestrial system would pass handily, since v << c and therefore γ is effectively 1 within the digits of precision required. However, the assumption that this is an accurate test would render GPS and other orbital navigational aids non-functional because of their high velocity and extreme timekeeping requirements (this is more an issue of the γ factor also being part of time dilation, which is likely the reason behind the apparent increase in mass, but I digress).

It is intellectually dishonest

AI models don't produce, they use various transforms to reassemble. Any code coming out was written by a human being, digested, and presented as a product of the LLM. Even if it was scraped from an open-source project, it is not provided under license, is not used in accordance with the license (as none is provided with the software) and has no attribution.

There is no accountability

Ignoring the legal requirements for accountability, if there's a derivative of someone else's code and they complain, the project can only point to a LLM that provided the information, and all derivative works are suspect. Using random code provided by any AI without proper attribution can seriously damage or kill a project.

A machine can never be accountable. Software can never be accountable. Only a person can be accountable. Who is responsible, by name, for any error in Copilot? I assume it's us, so it's the tool that's too hard to be used, and thus we're better off with real bespoke code and testing.

Claims of inevitability and insults aren't effective

Now, consistency and appeal to authority may work on suckers, but here's the thing, you talk about how automobiles were resisted, but you fail to consider thalidomide, asbestos, and radium clocks, all wonders of their age with downstream consequences with billions or more in damages and entire statutes based around their restriction.

And appeal to the inevitable? Come on, make solid arguments, don't vaguely Roko's Basilisk us into thinking this is fait accompli so we need to join you in this brave new world of free services that you just happen to advertise to us in this ham-handed way.

Anyway, I don't mean to make this about you, despite your attempt to make this about us. Here are some reasons, I'm sure there are plenty more. I'm similarly adamant in saying "no" to any of this nonsense, but you keep doing what it takes so you can eat.

Please excuse the length, it's Christmas Eve and I don't have the time to go through to proof and edit this post. Merry Christmas and Happy Holidays.

[RFnexus]

Hi @xa4hf8,

The problem with ChatGPT, Claude, et al, is that these models are not "AI". They are very advanced token predictors. It is the result of what happens when you feed the entirety of the Internet (and more) to something and ask it to predict the next word based on the text entered. That is what these models do. And therein lies the problem.

They do not perform any functions outside of predicting the next token in a sentence. And we humans tend to anthropomorphize the output because it resembles natural language. It's by both sheer luck and the sheer scale of data that these models can "write" code at all.

The model has no high level understanding of what the output it gives actually is. It's a complete black box. When you have a codebase like Reticulum that has thousands of moving parts and interconnected dependencies, it's just not something you can feed to a LLM and expect it to not break something. The technology isn't there yet despite the hype cycle.

Current LLMs have something called a context length. This is the maximum number of tokens you can feed it before the model performance degrades or it reaches a computational limit. A model like OpenAI's "GPT-4" has a context length of 128,000 tokens. The average codebase far exceeds this context length. No currently available LLM model can work fully within a project context and it's dependencies. And it doesn't look like the one you linked solves that problem either.

The YouTube channel 3Blue1brown has an amazing video that goes over what is actually happening inside these LLMs that can be understood with high school level algebra. The foundations that these models are built on are not actually that complex. And the technology has been around for quite awhile. The difference now is just the vast sea of data that exists and the level of compute we now have access to. Also, money. OpenAI probably would not have existed without the several billions that were thrown at it. They made a bet on the attention architecture and it paid off tremendously.

On a related note... I stumbled across this interesting article which indicates that physicians are refusing to incorporate A.I. into their practice even though statistics prove that failure to do so produces worse outcomes for patients. Key points:

I find this to be very sensationalist. ChatGPT or similar models should absolutely not be used for medical diagnosis. The main driver behind some of these scores like in the publication that article talks about is because that data was already present in it's training set. When GPT-4 first came out, peple were talking about how it passed the US Bar Exam and MCAT in the 90th percentile. The reason being is many of those questions and their answers were already present in the dataset used to train ChatGPT. Think about how many study guides for something like that out there are that exist.

When a model is presented with completely novel information the performance is likely to degrade. LLMs also do not have a ground truth, or the ability to verify the information it gives. This leads to the "hallucination" problem where models simply make things up, because that is what they do. They are token predictors. There is no high level thinking or anything like that which goes on.

And there is also a disturbing trend of entirely manipulating the scoring of model performance (See: "Re-evaluating GPT-4’s bar exam performance"). Remember that ChatGPT is a commercial product. They have to sell it to make money.

That's not to say there aren't valid uses of something like ChatGPT or other LLMs. As a quick tool to find documentation on something they excel at that task. Internet search is such complete shit that it's almost a necessity at this point. I have Qwen2.5 running on my own local machine and just yesterday it saved me the headache of searching how the command to turn on a WiFi hotspot using nmcli.

But as a total replacement to programming or any other task, it's not there. The appropriate understanding of LLMs should be "grep on crack".

Another issue is the tech debt that tools like ChatGPT or Copilot create. In personal projects this may be less of an issue, but in an enterprise or open source setting this absolutely will not work out. If you can't understand every piece of code that it wrote, it will lead to problems.

All I know is that previous technological advancements like automobiles & sewing machines were resisted in a similar fashion and none of those resistance movements succeeded. I would like to understand your reasoning here

This is a false equivalency that I've seen used for other things too. Cryptobros love this one. The impact of LLMs is currently nowhere near the impact that automobiles or sewing machines had. I've seen this same argument presented for "de-fi" in a very similar online discussion, yet I still to this day have no idea what it does.

and will continue to improve

There have been boom and bust cycles in machine learning research before. that are largely driven by market factors. I think the return on something like ChatGPT or similar LLMs is very marginal to the average end user, and I think that LLMs are reaching a plateau. This is of course a personal subjective opinion, so take it with a grain of salt. But I don't think the sensationalism and hysterics on both side of the debate is helping. I personally would love to be wrong. I think that AGI would be one of the biggest benefits to humanity ever. But for now, that remains science fiction.

And finally I will address this point

Anyway, since Microsoft just launched a free service tier for Github Copilot, I don't see how you can avoid it.

Is this really who you want to give your data to? There are insane amounts of privacy and safety issues already when it comes to cloud hosted services. Corporations have nothing but an adversarial relationship to the user. They do not give a shit about you and will do everything to f you over, sometimes just because they can. You have no autonomy or control over where your data is going. Going back to the article you mentioned regarding how "doctors should be using ChatGPT". Do you know how pissed I, as a patient would be if my sensitive medical information was being sent up to be compiled into OpenAI's training data so they can sell a product?

There's already been issues where CoPilot has leaked sensitive API keys before. Now imagine if this was your medical data. Or internal emails. There is a reason hundreds of companies block access to ChatGPT. Even though this is an entirely different discussion, I cannot stress the importance of how important it is that LLMs be open source, not connected to the Internet, and on-premise. There is only one future where humanity benefits from AI and that is the open source one.

[faragher]

Is this really who you want to give your data to?

Valid question, but the larger question is "if you need an account and EULA to use it, what legal rights are you giving up, and can you reasonably back out of the relationship when the terms of the deal change?"

[kc1awv]

I must admit that I am not expecting a sincere explanation (if one is even forthcoming at all.)

And, you're not owed one. This statement is bait, and you're fishing in the wrong pond.

Anyway, since Microsoft just launched a free service tier for Github Copilot, I don't see how you can avoid it.

Easy, just don't use it. Or rather, don't use it with the intention of being able to write valid novel code. Something, something, right tool for the job, something.

Identify, design, write, test, collaborate, resolve.

[xa4hf8]

This statement is bait, and you're fishing in the wrong pond.

It was actually more like a reaction to the gruff and terse original response. I never seriously expected anyone here to rely heavily on what Google & Microsoft are offering, but did think that some of the functionality could be useful in some contexts.

The screenshot I shared was behind a login, and I copied it to save others the trouble of creating an account to read what it said. There is merit in all of these posts, and I am not entirely unfamiliar with those objections -- but I appreciate the efforts to provide more detailed, subjective and specific characterization (and I reckon that others will too.) There is a lot that I can agree with, and some that I don't, but it's probably out of scope and not productive to debate farther here. Thanks for the discussion and happy holidays, folks.