Python sample (scripting ML/AI sample and web Django and Flask)

[moljac]

Sample integrating Aspire and Python (as Process/ExecutableResource - not Container)

• simple Python AI/ML script
• web frameworks
  • Django
  • Flask

Video

https://www.youtube.com/watch?v=oDSW0t_x9MM

Plans:

1. add containerized versions

Extended sample[s]

https://github.com/Samples-Playgrounds/Samples.Aspire/tree/main/samples/aspire-starter

Motivation - Based on feedback/discussion

@luuk777w
1 month ago

Currently working on a large distributed api, which mainly uses .NET of course, but also heavily relies on tools and services running in python and Matlab. For this, we do use .NET Aspire, and the experience so far has been great. The project is due to complete in 2 years, so the preview versions are good right now.

@harezalex
3 months ago

I'm skeptical. How would it work for heterogenous systems (here I mean systems with components created using different programming languages/technologies)? Let's say I have a FE written in js/ts and not blazor? Or I have a BE component(s) written in go/ruby/rust/python/anything else? Also, as mentioned here multiple times, it will work painlessly only for monorepos.

[moljac]

seems like CI does not install aspire workload

Error: D:\a\aspire-samples\aspire-samples\dotnet\sdk\8.0.200\Sdks\Microsoft.NET.Sdk\targets\Microsoft.NET.Sdk.ImportWorkloads.targets(38,5): error NETSDK1147: To build this project, the following workloads must be installed: aspire [D:\a\aspire-samples\aspire-samples\samples\AspireWithJavaScript\AspireJavaScript.AppHost\AspireJavaScript.AppHost.csproj]

[luuk777w]

I still wonder if this could work "on request".

Take a look at this python example:
https://github.com/luuk777w/horizon-perceive-canny-edge-tool-example

This is a very basic tool which does canny edge detection, and uses gRPC for communication.
Then, in my AppHost I just add it as a container, and have .NET minimalAPI service to expose REST endpoints and communicate with the tool.

var cannyEdgeToolContainer = builder
    .AddContainer("Canny Edge Detection Tool", "canny-edge-tool")
    .WithServiceBinding(hostPort: 50051, containerPort: 50051, name: "canny-edge-tool-endpoint");

var cannyEdgeToolEndpoint = cannyEdgeToolContainer
    .GetEndpoint("canny-edge-tool-endpoint");

var cannyEdgeService = builder
    .AddProject<Projects.PerceiveAPI_CannyEdgeService_API>("cannyedgeservice-api")
    .WithReference(cannyEdgeToolEndpoint);

So, could this also work, when it isn't a container, and there isn't gRPC, and it is just an executable? Of course, I can invoke "the normal way" (Process.Start()), but I mean the Aspire way, if there is.

[moljac]

I still wonder if this could work "on request".

Well, I am not cloud expert. Just using and learning along the way. I came from Aspire and MAUI (and other clients) integration initiative which sprung to my mind during internal presentation on Aspire and I asked David whether that would work for MAUI and MAUI is several clients in one.

Basically "on request" is feature used when orchestrator needs to scale (up/down, in/out), so I think I could make safe bet that Aspire team has that on their plate.

Take a look at this python example: https://github.com/luuk777w/horizon-perceive-canny-edge-tool-example

OK. I will. It will take some time.

This is a very basic tool which does canny edge detection, and uses gRPC for communication. Then, in my AppHost I just add it as a container, and have .NET minimalAPI service to expose REST endpoints and communicate with the tool.

var cannyEdgeToolContainer = builder
    .AddContainer("Canny Edge Detection Tool", "canny-edge-tool")
    .WithServiceBinding(hostPort: 50051, containerPort: 50051, name: "canny-edge-tool-endpoint");

var cannyEdgeToolEndpoint = cannyEdgeToolContainer
    .GetEndpoint("canny-edge-tool-endpoint");

var cannyEdgeService = builder
    .AddProject<Projects.PerceiveAPI_CannyEdgeService_API>("cannyedgeservice-api")
    .WithReference(cannyEdgeToolEndpoint);

So, could this also work, when it isn't a container, and there isn't gRPC, and it is just an executable?

I think it should. I have few ideas, but again it will take some time.

Right now I am trying to solve other, similar issues for clients... See even folder is called Client, because I just changed simple python scripting AI/ML sample to launch Django/Flask.

Of course, I can invoke "the normal way" (Process.Start()), but I mean the Aspire way, if there is.

Aspire is in preview and most likely - not eveything we (You and me) need is implemented.

I found out that Aspire SDK can easily be extended and this is what I am exploring and implementing along the way.

[luuk777w]

Okay, I worked out a more complete example, also including .NET code, so you have everything.
https://github.com/luuk777w/horizon-perceive-aspire-python-sample

This is how we currently do it. There is a Tool, developed by a research-orientated organization, which needs to be used by the API. So, now this tools runs in a container, hosting a gRPC server to handle requests. In the sample, if you look in tools you will find a dummy tool which does basic canny edge detection. Then, in source, you will find the accompanied .NET service which exposes an endpoint.

For others reading this: the reason for this complexity it because in academia, researchers have no clue whatsoever when it comes to system/cloud/web development. They mainly develop in MATLAB, and python. Therefore, we cannot just ask them to "expose some endpoints", or basically write a whole microservice in Python. They do not understand. Therefore, we have chosen the gRPC approach for now. However, this comes with its limitations and challenges as well, since gRPC isn't supported by MATLAB for example. It also requires quite some guidance from our part.

This makes us especially eager to learn to see if it is possible, and feasible, to run the tools as executables/processes. Then it would be possible to have it as an executable omitting gRPC and docker.

[moljac]

@DamianEdwards

1. Seems like errors are WASM related.
2. Do you want me to fix

   warning NU1506: Duplicate 'PackageVersion' items found. Remove the duplicate items or use the Update functionality to ensure a consistent restore behavior.
   

   ??
   In other projects. I can open other PR for that.

Edit:

Seems like on windows nuget restore is being canceled because of NU1506 (item 2 in list above)

[moljac]

Okay, I worked out a more complete example, also including .NET code, so you have everything. https://github.com/luuk777w/horizon-perceive-aspire-python-sample

OK. I took a sneak peek into the sample.

This is how we currently do it. There is a Tool, developed by a research-orientated organization, which needs to be used by the API. So, now this tools runs in a container, hosting a gRPC server to handle requests.

Is gRPC server is control manager? To start tool on demand.

If yes - this way Aspire.AppHost has no idea that tool (in Aspire lang - Resource, ExecutableResource or ContainerResource) was restarted.

I added embbeded HTTP server (watson) to Aspire-Python implementation and am able to hit it with request and Start.Process(). This is where I stopped. I need to test and add more features and there is still tons of missing nice-to-have features like

1. connecting to AppHost (connecting - passing data to and from)
2. connecting to Dashboard

In the sample, if you look in tools you will find a dummy tool which does basic canny edge detection. Then, in source, you will find the accompanied .NET service which exposes an endpoint.

Endpoint is gRPC endpoint of the "control manager". Am I right?

For others reading this: the reason for this complexity it because in academia, researchers have no clue whatsoever when it comes to system/cloud/web development. They mainly develop in MATLAB, and python. Therefore, we cannot just ask them to "expose some endpoints", or basically write a whole microservice in Python. They do not understand. Therefore, we have chosen the gRPC approach for now. However, this comes with its limitations and challenges as well, since gRPC isn't supported by MATLAB for example. It also requires quite some guidance from our part.

That is the reason I went with HTTP Watson 1st. Almost every data science language has HTTP client, but not server implementation and then Matlab script could request http://localhost:58589/python-endpoint-for-some-script/start/ and start python script on the other node (Aspire resource).

This makes us especially eager to learn to see if it is possible, and feasible, to run the tools as executables/processes.

Then it would be possible to have it as an executable omitting gRPC and docker.

I went with process (Process.Start()) approach 1st, because it is similar to MAUI integration and hooking in debugging could also be easier (not sure) and as number 1 - scientists and docker - no way. Apps are no-go zone for most of them.

I will take a deeper look and finish my sample... And then we can talk. Besides I have tons of questions for Aspire team.

[luuk777w]

Is gRPC server is control manager? To start tool on demand.

What do you exactly mean with "control manager"?
Right now, the gRPC server is constantly running. If you look in the tools folder, you will find all the gRPC fluff, and the actual tool is https://github.com/luuk777w/horizon-perceive-aspire-python-sample/blob/main/tools/canny_edge_detector.py

If yes - this way Aspire.AppHost has no idea that tool (in Aspire lang - Resource, ExecutableResource or ContainerResource) was restarted.

Well, the tool is just constantly running.

I added embbeded HTTP server (watson) to Aspire-Python implementation and am able to hit it with request and Start.Process(). This is where I stopped. I need to test and add more features and there is still tons of missing nice-to-have features like

How will this work? And if you are using an HTTP server, how does it differ from using a gRPC server? To me it doesn't solve the issue, or I might be overlooking something / not understanding it fully.

Endpoint is gRPC endpoint of the "control manager". Am I right?

I fail to understand the meaning of control manager in this context, maybe I am missing something though.

I went with process (Process.Start()) approach 1st, because it is similar to MAUI integration and hooking in debugging could also be easier (not sure) and as number 1 - scientists and docker - no way. Apps are no-go zone for most of them.

Yes, exactly.

---

However, the more I think about this, I can't really see a way for the tool to not be a continuous running process (e,g. gRPC server). Let's go to my sample, this is like the basic sequence:

 ┌────┐     ┌───────────────────┐ ┌─────────────┐
 │User│     │.NET WebAPI Service│ │Tool (python)│
 └─┬──┘     └─────────┬─────────┘ └──────┬──────┘
   │                  │                  │       
   │makes HTTP request│                  │       
   │─────────────────>│                  │       
   │                  │                  │       
   │                  │makes gRPC call to│       
   │                  │─────────────────>│       
   │                  │                  │       
   │                  │  returns result  │       
   │                  │<─────────────────│       
   │                  │                  │       
   │  returns result  │                  │       
   │<─────────────────│                  │       
 ┌─┴──┐     ┌─────────┴─────────┐ ┌──────┴──────┐
 │User│     │.NET WebAPI Service│ │Tool (python)│
 └────┘     └───────────────────┘ └─────────────┘

If the .NET API would just start the process on request (Process.Start()), it would take waaay too long and probably would not be very scalable. Right now, even with proper Aspire support I don't really see a way around this, besides, I also think this might be out of the scope of Aspire?

[moljac]

What do you exactly mean with "control manager"?

Naming is hard: control/command manager. Receives message/function-call/request and does something.

Let's make few steps back.

From what I see: gRPC server listens and when it receives something it starts tool (detail: for edge detection).

Is that right? Correct me if I am wrong.

And then we can go further...

[luuk777w]

Naming is hard: control/command manager. Receives message/function-call/request and does something.

Aahhh yes.

From what I see: gRPC server listens and when it receives something it starts tool (detail: for edge detection).

Yes thats correct. Are you familiar with gRPC? If not, you can read up on it here: https://grpc.io/
It's akin to an HTTP server, but instead of HTTP requests, they are Remote Procedure Calls (over HTTP2)

[moljac]

From what I see: gRPC server listens and when it receives something it starts tool (detail: for edge detection).

Yes thats correct.

OK. We are on the same page.

Are you familiar with gRPC?

I use it. When I learn I like to do practical things 1st and then dive into theory. So, with gRPC I am junior.

If not, you can read up on it here: https://grpc.io/ It's akin to an HTTP server, but instead of HTTP requests, they are Remote Procedure Calls (over HTTP2)

Thanks.

Let me finish my daily jobs which piled up because of conference and problems that were not under my control. And then I will finish sample with "Control Manager" I have in my dreams.

[DamianEdwards]

Thanks for this. However we'd like to keep this samples repo limited to demonstrations of Aspire hosting and components owned and shipped from the actual Aspire repo. I think the best place for samples related to community Aspire extensions is in the repos of those extensions.