Skip to content

Commit

Permalink
Documentation (#41)
Browse files Browse the repository at this point in the history
Add documentation and improve API
  • Loading branch information
Pierre-Yves authored Jan 29, 2024
1 parent 4966a85 commit 1aedfba
Show file tree
Hide file tree
Showing 7 changed files with 154 additions and 53 deletions.
180 changes: 128 additions & 52 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,79 +2,118 @@
[![](https://img.shields.io/badge/Community%20Extension-An%20open%20source%20community%20maintained%20project-FF4700)](https://github.com/camunda-community-hub/community)
![Compatible with: Camunda 8](https://img.shields.io/badge/Compatible%20with-Camunda%208-0072Ce)

Library to manipulate File process variable, and store the content in different storage (Folder, CMIS...)
Library to manipulate File process variables and store the content in different storage (Folder, CMIS...)

This library is used by connectors.
This library is used by connectors or by an application.

# Principle
Zeebe can't manipulate the files, only String as a process variable.

# Store file to the storage
The idea is to give to the File Storage the file and get back a **File Reference**.
The **File Reference** can be saved as a process variable.

Get an instance of the factory
## Store a file

````
Multiple storage are available: to specify which file storage is used, a **Storage Definition** is provided.
![Store file](doc/StoreFile.png)

Multiple storages are available (see the section below).

* CMIS: Any Content Management Interoperability Services. A lot of tools exist.
* JSON: the file is saved directly in the file reference. Zeebe allows only 4 Mb for the payload; this method works only for small files.
* Folder: a folder on the host. Specify a shared folder in the cluster
* Temporary Folder: the temporary folder of the host is used (assuming all applications/connectors/workers run on the same host)


## Read a file
With the *File Reference**, it is possible to retrieve the file

![Read file](doc/ReadFile.png)

Note: The storage "URL" can be used to read a file from a URL. It is a Read Only storage.



## Purge a file
The file can be purged from the storage when the process is finished. The **File Reference** is provided.

There is no "expiration mechanism" (except the temporary folder, which is purged on each reboot), so the application must purge files.

![Purge file](doc/PurgeFile.png)

# API

The API contains three main access.

## Store files

Get an instance of the factory.

````java
FileRepoFactory fileRepoFactory = FileRepoFactory.getInstance();
````

Create a FileVariable. Upload the file in the FileVariable object
Create a **FileVariable**. Upload the file in the **FileVariable** object.

````
````java
FileVariable fileVariable = new FileVariable();
byte[] contentInByte = <Content In Byte>
fileVariable.setValue( contentInByte );
fileVariable.setName( "MyDocumentName.pdf");
fileVariable.setMimeType( FileVariable.getMimeTypeFromName(fileVariable.getName()));
`````

Set the Storage Definition. The Storage definition describe where the core document is stored.
Set the **Storage Definition**. The **Storage Definition** describes where the core document is stored.


````
````java
StorageDefinition storageDefinition = StorageDefinition.getFromString(storageDefinitionSt);

fileVariable.setStorageDefinition(storageDefinition);
````

See below the different Storage definition available
See below the different Storage definitions available, and the structure of the storageDefinitionSt.


Save the File Variable. The function return a FileVariableReference
Save the **File Variable**. The function returns a **FileVariableReference**.


````
````java
FileVariableReference fileVariableReference = fileRepoFactory.saveFileVariable(fileVariableValue);
````

The fileVariableReference is only a reference, and can be saved in the process variable
The fileVariableReference is only a reference and can be saved in the process variable.

````
String processVariableSt = fileVariableReference.toJson()
````java
String processVariableSt = fileVariableReference.toJson();
// save the processVariableSt as a process variable
````

# Read file from the storage
## Read a file

To read a file from the storage, the first operation consist of accessing the FileVariableReference
Access the file from the **FileVariableReference**.

````
````java
String processVariableSt = <GetFromProcessVariable>
FileVariableReference fileVariableReference = FileVariableReference.fromJson( processVariableSt );
````

Then get the FileVariable from the repository
Then, get the **FileVariable** from the repository.

````
````java
FileRepoFactory fileRepoFactory = FileRepoFactory.getInstance();
FileVariable fileVariable = fileRepoFactory.loadFileVariable(fileVariableReference)
FileVariable fileVariable = fileRepoFactory.loadFileVariable(fileVariableReference);
````

Content, fileName, MimeType are available via method.
Content, fileName, and MimeType are available via the method.

The storage definition is stored in the FileVariable, so to update it, change the content and store it again.
The **Storage Definition** information is saved in the **FileVariable** to update it, change the content, and store it again.

# Purge a fileVariable
From the FileVariableReference, it is possible to purge the document in the storage.
## Purge a file

````
From the **FileVariableReference**, purging the document in the storage is possible.

````java
String processVariableSt = <GetFromProcessVariable>
FileVariableReference fileVariableReference = FileVariableReference.fromJson( processVariableSt );

Expand All @@ -83,71 +122,108 @@ From the FileVariableReference, it is possible to purge the document in the stor
````

# Storage definition
Different storage definition are available. The storage store the core of the document.
Different *Storage Definition** are available. The storage stores the core of the document.

## JSON

The file is saved in JSON, and the FileVariableReference contains the complete document. This is very efficient, but Zeebe limited the size of a process to 4 Mb. This solution is not possible to saved documents.
The file is saved in JSON, and the **FileVariableReference** contains the complete document.
It is very efficient, but Zeebe limited the size of the process to 4 Mb.
This solution does not make it possible to save documents (Word, OpenOffice, PDF) which are in general bigger.

Storage Definition Key: "JSON"

````
````java
import StorageJSON;

StorageCMIS.getStorageDefinitionString()
String storageDefinition = StorageJSON.getStorageDefinitionString();
````


## Temporary folder

The file is saved in the Temporary folder of the host.
If multiple applications (different connectors) needs to access the file, this is not an acceptable document, except if all application are hosted by the same machine (or the same Pod)
The file is saved in the Temporary folder of the host.
If multiple applications (different connectors) need to access the file, this is not an acceptable document, except if all applications are hosted by the same machine (or the same Pod)

Storage Definition Key: "TEMPFOLDER"

````
````java
import StorageTempFolder;

StorageCMIS.getStorageDefinitionString()
String storageDefinition = StorageTempFolder.getStorageDefinitionString();
````

## Folder
The file is saved in the folder given in the connection string.
If multiple applications needs to access the file, the folder must be visible and share on the same place (/mnt/filestoreage" for example).
The file is saved in the folder given in the connection string.
If multiple applications need to access the file, the folder must be visible and shared in the same place (/mnt/file storage" for example).

Storage Definition Key: "FOLDER:/mnt/filestorage"
Storage Definition Key: "FOLDER:/this/is/the/path/to/the/storage"


````
````java
import StorageFolder;

StorageCMIS.getStorageDefinitionString(String folder)
String storageDefinition = StorageFolder.getStorageDefinitionString(String folder);
````


## CMIS

The file is saved in a CMIS tool. The connection to the tool (Url, Repository name, username, password) must be provide.
The folder where the file must be store must be provide too.
The file is saved in a CMIS tool. The connection to the tool (URL, Repository name, username, password) must be provided.
The folder where the file must be stored must be provided, too.

Storage Definition Key: "CMIS:{"url":"<url>", "repositoryName":"<repositoryName>", "userName": "<userName>", "password": "<password>", "storageDefinitionFolder":"<folder>"}"

The static methode is available in the StorageCMIS class
The static method is available in the StorageCMIS class.

````
````java
import StorageCMIS;

StorageCMIS.getStorageDefinitionString(String url, String repositoryName, String userName, String password, String storageDefinitionFolder)
String storageDefinition = StorageCMIS.getStorageDefinitionString(String url, String repositoryName, String userName, String password, String storageDefinitionFolder)
````

## URL
The file may be access via an URL. This storage is use to READ only.
The file may be accessed via a URL. This storage is used to READ only.

For example, this source file reads a document from a URL.
````java
import StorageURL;

String storageDefinition = StorageURL.getStorageDefinitionString("https://github.com/camunda-community-hub/camunda-8-connector-officetopdf/raw/main/src/test/resources/OfficeToPdfExample.docx");
````

## Get from a String

This method return a Storage definition from a String

````java
import StorageDefinition;

StorageDefinition storageDefinition = StorageDefinition.getFromString( storageDefinitionSt );

````

For example, this soure file read a document from an URL.
`````json
{
"storageDefinition": "URL",
"content": "https://github.com/camunda-community-hub/camunda-8-connector-officetopdf/raw/main/src/test/resources/OfficeToPdfExample.docx"
}
The format of the string is the following:
<Type>:<Complement>

`````
Types are **JSON**, **TEMPFOLDER**, **FOLDER**, **CMIS**, **URL**

Complement depends of the type.

**JSON**, **TEMPFOLDER**, **URL** does not have any complement.

**CMIS** contains as reference the access information, saved in JSON, for example
````
"CMIS:{\"url\":\"http://localhost:8099/lightweightcmis/browser\",\"userName\":\"cmisaccess\",\"password\":\"demo\",\"storageDefinitionFolder\":\"/storagecmis\"}"
````


*FOLDER* needs to describe the folder
````
"FOLDER:/this/is/the/path/to/the/storage"
````

From a *StorageDefinition*, the string can be get:

`````java
String storageDefinitionSt = storageDefinition.encodeToString()
`````
Binary file added doc/PurgeFile.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added doc/ReadFile.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added doc/StoreFile.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
2 changes: 1 addition & 1 deletion pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@

<groupId>io.camunda.filestorage</groupId>
<artifactId>filestorage</artifactId>
<version>1.2.0</version>
<version>1.2.1</version>

<properties>
<maven.compiler.release>17</maven.compiler.release>
Expand Down
9 changes: 9 additions & 0 deletions src/main/java/io/camunda/filestorage/StorageFolder.java
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,15 @@ public static String getStorageDefinitionString(String folder ) {
return StorageDefinition.StorageDefinitionType.FOLDER.toString()+StorageDefinition.STORAGE_DEFINITION_DELIMITATEUR+folder;
}

/**
*
* @param folder folder to save the file
* @return the connection string
*/
public static String getStorageDefinitionString(Path folder ) {
return StorageDefinition.StorageDefinitionType.FOLDER.toString()+StorageDefinition.STORAGE_DEFINITION_DELIMITATEUR+folder.toString();
}

/**
* Save the file Variable structure in the temporary folder
*
Expand Down
16 changes: 16 additions & 0 deletions src/main/java/io/camunda/filestorage/StorageURL.java
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,22 @@ protected StorageURL(StorageDefinition storageDefinition, FileRepoFactory fileRe
super(storageDefinition, fileRepoFactory);
}

/**
*
* @param url to access
* @return the connection string
*/
public static String getStorageDefinitionString(URL url ) {
return StorageDefinition.StorageDefinitionType.URL.toString()+StorageDefinition.STORAGE_DEFINITION_DELIMITATEUR+url.toString();
}
/**
*
* @param url to access
* @return the connection string
*/
public static String getStorageDefinitionString(String url ) {
return StorageDefinition.StorageDefinitionType.URL.toString()+StorageDefinition.STORAGE_DEFINITION_DELIMITATEUR+url;
}
@Override
public String getName() {
return "URL";
Expand Down

0 comments on commit 1aedfba

Please sign in to comment.