How to use - Readme missing

[SirSundays]

Hi @paul121,
it's really great what you did here already. First I wanted to write my own package for adding an influxdb datastream but when I found this it was obsolete to do a second one (and I'm also a beginner in PHP/farmOS/drupal so...).

I got this one here installed and got a connection to my influxdb but what I was wondering: What next?
How do I add an influxdb datastream to a sensor for example? Would I need a special influxdb sensor for that (no right?)?
I hope this isn't a too silly question.

Maybe if you can help me along with the next steps that would be great.

Jan

[paul121]

Hey @SirSundays !! Thanks for reaching out and opening this issue. I agree a readme is needed... this module is still a bit of work in progress though 😃 Any input you have is most certainly welcome!

As you've noticed the module itself doesn't provide an actual datastream type (yet?). Right now it is only providing some higher level pieces that (I hope) will make it easier to integrate data streams with Influx db:

• A UI for configuring connections to influx server(s) - it seems that you tested this config UI - how did it go?
• An initial wrapper around the Influx2 PHP client to make querying data from influx easier: https://github.com/Vital-Agronomics/farm_influxdb/blob/2.x/src/InfluxdbServerClient.php
• For data streams, an abstract base class to extend from: https://github.com/Vital-Agronomics/farm_influxdb/blob/2.x/src/Plugin/DataStream/DataStreamType/InfluxdbBase.php#L14

My intention is to build out some helper methods on both the InfluxdbServerClient and InfluxdbBase classes to make it easier to query Influx for data associated with a data stream. This association is the main complexity here and reason why it's hard to provide a "default" influx db data stream type with this module that will work for everyone.

For example, will your data in influx db be tagged with the same UUID as the data stream UUID? Or are multiple tags required to uniquely identify data in influx db? And does this mean your data stream requires additional field(s) in the farmOS DB to save this configuration?

Additionally, how should the data be queried? Since influx db is a time series db there are many more options for querying and aggregating data. Depending on the type of data/sensor it might make sense to aggregate over a given internval instead of returning each data point.

At least initially it seems too complicated to create a "default" influx data stream that can work for different use cases. I'd love to make all of these things configurable from the UI but right now it's less work to hard-code these behaviors in separate data stream classes.

[paul121]

Oh, but also worth noting, this project got put on the back burner while turning focus to core farmOS 2.x work. But farmOS 2.x is coming along well and should have a beta release soon - so I will be giving this module some attention again soon! Expect to see some updates (and a readme!) next soon, hopefully next week..

[SirSundays]

Hi, @paul121 thx for your answer.

it seems that you tested this config UI - how did it go?

so after installing the plugin (copying all the files to sites/all/modules/farm I think and installing it in farmOS itself) I first got the error of invalid URL, because I forgot the HTTP. Then I wanted to test the connection but wasn't able because I didn't install the influx lib beforehand manually. I thought it would do it all automatically in the background. After that, I could successfully test the connection (was good).

My intention is to build out some helper methods on both the InfluxdbServerClient and InfluxdbBase classes to make it easier to query Influx for data associated with a data stream.

So I got it right that Step one is basically to make the library easily modifiable so one can customize it after their likings?

For example, will your data in influx DB be tagged with the same UUID as the data stream UUID?

Maybe I go to this topic a bit more naive, but I think it wouldn't need the same UUID. If you can configure the datastream and tell it please get the data from this measurement (+ maybe the specific field n the measurement) it should be fine.

A bit of a background for you: I'm working with most of my sensors through Node-Red and saving all my data in InfluxDB. In Node-Red I just have the raw value and say: put this value into this influxdb database under this measurement. So very simple. Right now I then display all the values in Grafana. (At the End this maybe should also be in farmOS. So that farmOS has management as well as monitoring)

Additionally, how should the data be queried? Since influx db is a time series db there are many more options for querying and aggregating data. Depending on the type of data/sensor it might make sense to aggregate over a given interval instead of returning each data point.

You are right. I think something like this needs to implement but not at a very high complexity. Maybe it could be similar to the implementation in Grafana (tho not as complex). There you can basically select group by interval or give me every value. And if you group them, should it then take min max mean etc.

I'd love to make all of these things configurable from the UI but right now it's less work to hard-code these behaviors in separate data stream classes.

I can completely understand that!

hopefully next week..

I'm really looking forward to that!

I really wish I could help more with that but these are my first steps in PHP and kinda have an idea of how such a module is build-up, but coming from nodejs this is sometimes still a bit confusing. Let's say I was really glad and happy when I was able to modify the NRCS module to have my own map in it 😂

[paul121]

Then I wanted to test the connection but wasn't able because I didn't install the influx lib beforehand manually. I thought it would do it all automatically in the background. After that, I could successfully test the connection (was good).

Ah yes.. sorry about that. I just updated the composer.json to have a dependency on the influxdb-client-php library: db2359d

So if you install this module with composer eg: composer require vital-agronomics/farm_influxdb it will install the library as well. Note that right now this means you need to configure a git repository in your project composer.json that points to this Github repository. Eventually, this module should be added as a project to drupal.org, in which case you could then use composer require drupal/farm_influxdb.

Maybe I go to this topic a bit more naive, but I think it wouldn't need the same UUID. If you can configure the datastream and tell it please get the data from this measurement (+ maybe the specific field n the measurement) it should be fine.

Totally! That's another valid use case as well. I think the reason for using tags is so that you can identify multiple sensors all posting soil_temperature measurements to influxdb. Does that make sense? At the end of the day there are lots of ways this could be sliced up.

I really wish I could help more with that but these are my first steps in PHP and kinda have an idea of how such a module is build-up, but coming from nodejs this is sometimes still a bit confusing. Let's say I was really glad and happy when I was able to modify the NRCS module to have my own map in it joy

No worries! Really happy to hear form you @SirSundays and welcome to the farmOS community! Great job figuring out a custom map!! That involves understanding a bit of everything. There certainly is a learning curve with Drupal and farmOS but lots of powerful things once you can get a grip for it.

Just want to share a couple things if you aren't already aware:

• The farmOS forum: https://farmos.discourse.group/
• Monthly call (next one in a couple days, 11/10): https://farmos.org/community/monthly-call/
• We also have started a weekly development call at 9AM PST on Thursdays: https://farmos.discourse.group/t/interest-in-an-ongoing-programming-development-call/756

[SirSundays]

Eventually, this module should be added as a project to drupal.org, in which case you could then use composer require drupal/farm_influxdb.

This will be the best-case scenario in the end for sure.

That's another valid use case as well. I think the reason for using tags is so that you can identify multiple sensors all posting soil_temperature measurements to influxdb.

Ah ok, now I get what you mean before. For me, this is really never a problem. As I said before I'm using Node-Red in such projects every time so when I have multiple sensors from the same kind then I will always give the measurement a distinctive name. So that I can differentiate them in any case.

Does that make sense?

Yes but I never thought of that to be honest 😂

There certainly is a learning curve with Drupal and farmOS but lots of powerful things once you can get a grip for it.

I hope I will get there in the next couple of months.

I had the next monthly calls already on my calendar but was not aware of the development calls, to be honest. I will definitely try to be there in some of the meetings maybe I will then get a broader view on all the things and see how they connect.
So I may see/hear you there!