Skip to content

Espruino networking

Jump to bottom Edit New page
Gordon Williams edited this page Feb 23, 2017 · 2 revisions

THIS WIKI IS OUT OF DATE. check out the README.md on GitHub for docs on Espruino internals, or see www.espruino.com for searchable, up to date docs on using Espruino.

The Espruino networking has an architecture model that appears to be implemented in network.c. Various network support types can be compiled into the core code including:

  • JSNETWORKTYPE_CC3000 - The CC3000 device.
  • JSNETWORKTYPE_W5500 - The W5500 device.
  • JSNETWORKTYPE_ESP8266 - A piggybacked ESP8266 attached to the MCU.
  • JSNETWORKTYPE_LINUX - The Linux OS
  • JSNETWORKTYPE_JS - Unknown

For the ESP8266 board project, we will create a new type called JSNETWORKTYPE_ESP8266_BOARD.

Among the core functions exposed we seem to have:

  • networkCreate() - create the network object (ONLY to be used by network drivers)
  • networkWasCreated()
  • networkGetFromVar()
  • networkGetFromVarIfOnline()
  • networkSet()
  • networkFree()
  • networkGetCurrent() - Get the currently active network structure. can be 0!
  • networkParseIPAddress()
  • networkParseMACAddress()
  • networkGetAddressAsString() - Get an address as a string.
  • networkPutAddressAsString()
  • networkFlipIPAddress()
  • networkGetHostByName() - Use this for getting the hostname, as it parses the name to see if it is an IP address first.

A logical data type called JsNetwork contains the entry points for many of the functions.

This includes:

  • createsocket - if host=0, creates a server socket otherwise creates a client socket (and automatically connects). Returns >=0 on success.
  • closesocket - destroys the given socket.
  • accept - If the given server socket can accept a connection, return it (or return < -1).
  • gethostbyname - Get an IP address from a name.
  • recv - Receive data if possible. returns nBytes on success, 0 on no data, or -1 on failure.
  • send - Send data if possible. returns nBytes on success, 0 on no data, or -1 on failure.
  • idle - Called on idle. Do any checks required for this device.
  • checkError - Call just before returning to idle loop. This checks for errors and tries to recover. Returns true if no errors.

Each of these functions must be implemented by a network provider.

##createsocket

int net_<board>_createSocket(JsNetwork *net, uint32_t ipAddress, unsigned short port)

If host=0, creates a server otherwise creates a client (and automatically connects).

  • net - The Network we are going to use to create the socket.
  • ipAddress - The address of the partner of the socket.
  • port - The port number that the partner is listening upon.

Returns >=0 on success and -1 on an error. The return is the new socket id.

##closesocket

Destroys the given socket.

void net_<board>_closeSocket(JsNetwork *net, int sckt)

  • net - The Network we are going to use to create the socket.
  • sckt - The socket to be closed.

##accept

If the given server socket can accept a connection, return it (or return < 0).

int net_<board>_accept(JsNetwork *net, int serverSckt)

  • net - The Network we are going to use to create the socket.
  • serverSckt - The socket that we are now going to start accepting requests upon.

Returns a new conversation socket or -1 on an error.

##gethostbyname

Get an IP address from a name. Sets outIp to 0 on failure.

void net_<board>_gethostbyname(JsNetwork *net, char *hostName, uint32_t *outIp)

  • net - The Network we are going to use to create the socket.
  • hostName - The string representing the hostname we wish to lookup.
  • outIp - The address into which the resolved IP address will be stored.

##recv

int net_<board>_recv(JsNetwork *net, int sckt, void *buf, size_t len)

  • net - The Network we are going to use to create the socket.
  • recv - The socket from which we are to receive data.
  • buf - The storage buffer into which we will receive data.
  • len - The length of the buffer.

Returns the number of bytes received which may be 0 and -1 if there was an error.

##send Send data if possible.

int net_<board>_send(JsNetwork *net, int sckt, const void *buf, size_t len)

  • net - The Network we are going to use to create the socket.
  • sckt - The socket over which we will send data.
  • buf - The buffer containing the data to be sent.
  • len - The length of data in the buffer to send.

Returns number of bytes sent on success, 0 on no data, or -1 on failure

##idle

void net_<board>_idle(JsNetwork *net)

  • net - The Network we are going to use to create the socket.

##checkError

bool net_<board>_checkError(JsNetwork *net)

  • net - The Network we are going to use to create the socket.

This function returns true if there are no errors.

#The Socket Server library

The socket server library is a suite of components used to provide a sockets like abstraction within the Espruino environment. In addition to pure communication, there is much function in here for being either an HTTP client or an HTTP server. To ensure that we understand what that later part means ... an HTTP client is playing the part of a browser and will send HTTP requests to and HTTP server such as a Web Server.

The HTTP server part of this library plays the part of being an HTTP server that responds to client requests from browsers (or other HTTP clients).

To make this technology work, the library created hidden variables in the JS variable root. These are real JS variables but they are not visible or available to normal JS programs. However, from an internals perspective, they can be worked with just fine. These JS variables (and there are a couple) are actually instances of JS arrays where each element of the array is an object that represents a connection to a partner over the network. When working with the arrays, there is a method called socketGetArray() that returns (and creates) the array for us so we don't have to work with the raw arrays ourselves as a consumer of the library.

The Socket class has the following additions to it:

  • data - Event
  • close - Event
  • drain - event
  • available - method - jswrap_stream_available
  • read - method - jswrap_stream_end
  • write - method - jswrap_jswrap_net_socket_write
  • pipe - method - jswrap_pipe

###httpAppendHeaders Add the HTTP headers object to the string.

static void httpAppendHeaders(JsVar *string, JsVar *headerObject)

The headersObject is a JS object with name/value properties. Here we walk through each of those properties and append them to the string supplied as string and add each one as an HTTP header of the form <name>: <value>.

###httpParseHeaders Parse out the HTTP headers from the data.

bool httpParseHeaders(JsVar **receiveData, JsVar *objectForData, bool isServer)

The receiveData is the address of a JS variable that contains the string received from the partner. It is parsed to look for headers. These are added as new property of the object passed in as objectForData. The name of the new property that is added is called headers. Special treatment is given for HTTP client vs server requests. If we are a server, then method and url are especially parsed and added and if we are a client then httpVersion, statusCode and statusMessage are pulled out.

###httpStringGet Get a C string from a JS Var.

size_t httpStringGet(JsVar *v, char *str, size_t len)

Retrieve a string from the variable v and store it at the buffer pointed to by str for a maximum size of len. It appears that there is no null terminator and that the return is confusing.

###socketGetArray Return the hidden variable array.

static JsVar *socketGetArray(const char *name, bool create)

A JS array is returned called name where it is a hidden root variable. If it doesn't exist then it can be optionally created.

###socketGetType Retrieve the type of the socket variable.

static SocketType socketGetType(JsVar *var)

Given a socket variable, retrieve its type. The property in the socket variable is currently type and is an integer encoded as:

  • 0 - ST_NORMAL
  • 1 - ST_HTTP

###socketSetType

Set the type of the socket variable.

static void socketSetType(JsVar *var, SocketType socketType)

Given a socket variable and its type, set the type on that socket variable. This sets a property that is currently called type and is an integer encoded as:

  • 0 - ST_NORMAL
  • 1 - ST_HTTP

###_socketConnectionKill Close (kill) the socket for a specific socket variable.

void _socketConnectionKill(JsNetwork *net, JsVar *connection)

Kill a socket connection that is the socket contained within connection.

###_socketCloseAllConnectionsFor

Close ALL the sockets associated with something.

static void _socketCloseAllConnectionsFor(JsNetwork *net, char *name)

Close all the sockets associated with a named hidden variable.

###_socketCloseAllConnections Close all the sockets associated with some socket sets.

static void _socketCloseAllConnections(JsNetwork *net)

Close all the sockets associated with the socket sets called HttpSC, HttpCC and HttpS.

###socketSendData Send data through the socket.

bool socketSendData(JsNetwork *net, JsVar *connection, int sckt, JsVar **sendData)

The implementation of this one is tricky. It appears that sendData must be a string as we perform string lengths against it. Can it hold binary? We also appear to pass in a connection which I thought would know its own socket ... but yet it takes a socket integer as a parameter as well.

The sendData is shrunk on return having pruned off what was actually sent. However, this rountine does not assure to send all the data ... just some data ... so a caller should not assume that once it is called, the data has been sent.

###socketInit Initialize the socket server environment.

void socketInit()

Only seems to do something on windows.()

###socketKill Shutdown the socket subsystem.

void socketKill(JsNetwork *net)

###socketServerConnectionsIdle Do idle processing on server connections.

bool socketServerConnectionsIdle(JsNetwork *net)

Walk through each of the socket connections for servers HTTP_ARRAY_HTTP_SERVER_CONNECTIONS (HttpSC) and look for properties on them that are interpreted as instructions.

Instructions that are processed include:

  • HTTP_NAME_CLOSENOW (closeNow) - Close a socket.
  • HTTP_NAME_SEND_DATA (dSnd) - Send data.
  • HTTP_NAME_CLOSE (close) - Close a socket when done.

###socketClientPushReceiveData Push received data into a stream.

void socketClientPushReceiveData(JsVar *connection, JsVar *socket, JsVar **receiveData)

###socketClientConnectionsIdle Do idle processing on HTTP client connections.

bool socketClientConnectionsIdle(JsNetwork *net)

Walk through each of the socket connections for HTTP clients HTTP_ARRAY_HTTP_CLIENT_CONNECTIONS (HttpCC) and look for properties on them that are interpreted as instructions.

  • HTTP_NAME_CLOSENOW (closeNow) - Close a socket.
  • HTTP_NAME_RECEIVE_DATA (dRcv) - Receive data.
  • HTTP_NAME_CLOSE (close) - Close a socket when done.

This function also polls the receive from the board.

###socketIdle Do idle processing on ??? connections.

bool socketIdle(JsNetwork *net)

Walk through each of the socket connections for ??? HTTP_ARRAY_HTTP_SERVERS (HttpS).

We examine the socket to see if there is a pending connection upon it.

###serverNew Create a new server socket.

JsVar *serverNew(SocketType socketType, JsVar *callback)

Create a new server socket of the given socketType (one of ST_NORMAL or ST_HTTP). Add the callback function as a property of the new socket variable. The property is HTTP_NAME_ON_CONNECT (#onconnect). Return the new socket variable. Note that this does NO network work other than create the socket JS var.

The variable returned is of class type httpSrv or Server depending on the socket type.

###serverListen

Start listening.

void serverListen(JsNetwork *net, JsVar *server, int port)

Create a real network connection for the socket JS variable supplied in server and cause it to start listening on the given port. The new socket is added to the list of server sockets HTTP_ARRAY_HTTP_SERVERS (HttpS).

###serverClose Close the server socket.

void serverClose(JsNetwork *net, JsVar *server)

Close the server socket given by the socket JS variable supplied in server.

###clientRequestNew

Create a new client socket.

JsVar *clientRequestNew(SocketType socketType, JsVar *options, JsVar *callback)

The new socket is added to the list of client sockets HTTP_ARRAY_HTTP_CLIENT_CONNECTIONS (HttpCC). We register the callback function named in callback with the HTTP_NAME_ON_CONNECT (#onconnect) method of the new socket variable.

The returned object is of class type httpCCRs, httpCRq or Socket depending on the socket type.

###clientRequestWrite Write some data to the client socket.

void clientRequestWrite(JsVar *httpClientReqVar, JsVar *data)

Write the data supplied by data to the client socket. This is implemented by adding the data to the property called HTTP_NAME_SEND_DATA (dSnd) property. The data is not actually sent by this function but is instead built and made available to be sent later.

###clientRequestConnect

Connect a socket to the server.

void clientRequestConnect(JsNetwork *net, JsVar *httpClientReqVar)

Connect a socket to the server. The variable supplies as a socket variable is expected to have a property on it called HTTP_NAME_OPTIONS_VAR (opt) which has a property called port which is the port number to connect with. In addition it should have a property called host that is the target host.

###clientRequestEnd

Signal the end of the socket.

void clientRequestEnd(JsNetwork *net, JsVar *httpClientReqVar)

Things get strange here based on the type of socket with which we are working. If the type is ST_HTTP we actually perform a connect request!!! For normal sockets, we register that we are ready to close after all the data has been sent. This is done by setting the HTTP_NAME_CLOSE (close) flag.

###serverResponseWriteHead

Set the HTTP response code and headers.

void serverResponseWriteHead(JsVar *httpServerResponseVar, int statusCode, JsVar *headers)

Record the HTTP response code and headers to be sent back to the partner.

###serverResponseWrite

Set the data to be sent back to the partner.

void serverResponseWrite(JsVar *httpServerResponseVar, JsVar *data)

Set the data to be sent back to the partner.

###serverResponseEnd

Write the data and flag the close of the connection.

void serverResponseEnd(JsVar *httpServerResponseVar)

Write the data and flag the close of the connection.

#ESP8266 Implementation Notes

It looks like we are going to have a problem with gethostbyname(). This appears to be a blocking call in Espruino while in the ESP8266 it is a callback function. The equivalent ESP8266 API is espconn_gethostbyname() which takes a callback that will be invoked when the host name is resolved.

#The net library

The net library brings together the network routines and the socket routines into a high level composed form.

  • jswrap_net_idle - Type: idle
  • jswrap_net_init - Type: init
  • jswrap_net_kill - Type: kill
  • jswrap_url_parse - Type: staticmethod, class: url
  • jswrap_net_createServer - Type: staticmethod, class: net, return: Server
  • jswrap_net_connect - Type: staticmethod, class: net, return: Socket
  • jswrap_net_server_listen - Type: method, class: Server
  • jswrap_net_server_close - Type: method, Class: Server
  • jswrap_net_socket_write - Type: method, Class: Socket
  • jswrap_net_socket_end - Type: method, Class: Socket
  • jswrap_stream_available - Type: method, Class: Socket
  • jswrap_stream_read - Type: method, Class: Socket

##jswrap_net_idle

Perform the idle processing.

bool jswrap_net_idle()

##jswrap_net_init

Initialize the network stuff.

void jswrap_net_init()

##jswrap_net_kill

Destory the network.

void jswrap_net_kill()

##jswrap_url_parse

Parse a URL string.

JsVar *jswrap_url_parse(JsVar *url, bool parseQuery)

The url variable is a String that is to be parsed. If parseQuery is true, we also parse any query data passed in. The object that is returned contains:

  • method
  • host
  • path
  • pathname
  • port
  • search
  • query

##jswrap_net_createServer

Create a new ST_NORMAL type server.

JsVar *jswrap_net_createServer(JsVar *callback)

Mostly a pass through to serverNew. The return type is a JS object of type Server.

##jswrap_net_connect

Form a connection to a client connection to a partner.

JsVar *jswrap_net_connect(JsVar *options, JsVar *callback, SocketType socketType)

If options is a string, then we parse it as a URL string otherwise it should be an object such as that returned by jswrap_url_parse.

The socket type is one of:

  • ST_NORMAL - 0
  • ST_HTTP - 1

The variable returned is a socket object. The logic in this function calls clientRequestNew and if we are not an ST_HTTP socket, then we call clientRequestConnect. The return is the object returned from clientRequestNew which means it is an instance of class type httpCCRs, httpCRq or Socket depending on the socket type.

##jswrap_net_server_listen

Start a server listening.

void jswrap_net_server_listen(JsVar *parent, int port)

Set the server listening. The port on which it will listen is supplied by port. This is mostly a pass through to serverListen.

This function is mapped to the following JavaScript methods:

  • httpSrv.listen()

##jswrap_net_server_close

void jswrap_net_server_close(JsVar *parent)

Close the server. This is mostly a pass through to serverClose.

##jswrap_net_socket_write

Write data through the socket.

bool jswrap_net_socket_write(JsVar *socketVar, JsVar *data)

The data is the new data to write. This is a pass through to clientRequestWrite.

##jswrap_net_socket_end

Close the socket with optional data.

void jswrap_net_socket_end(JsVar *parent, JsVar *data)

This is a pass through to clientRequestEnd.

#The HTTP Subsystem A library is provided that allows Espruino to perform HTTP services. There are two flavors to this. The first is that the Espruino can behave as an HTTP client (i.e. a browser or REST caller) and transmit and receive HTTP request.

Secondly, the Espruino can become an HTTP server and listen for incoming requests from browsers or REST clients.

To be a server, one would use:

var http = require("http");
var httpSrv = http.createServer(function(request, response) {
	// We have a new request here!!
});
httpSrv.listen(80);

Should we wish the server to stop listening for new incomming connections we can call httpSrv.close().

On the other side of the coin, we may wish the ESP8266 to be an HTTP client making HTTP requests in the same fashion as a browser or as a REST client.

We can perform a

http.get({
	host: "ipAddress",
	port: <portNumber>
}, function(response) {
	// Handle response
});

The HTTP subsystem is implemented in jswrap_http.c.

  • jswrap_http_createServer
  • jswrap_http_get
  • jswrap_httpSRs_write
  • jswrap_httpSRs_end
  • jswrap_httpSRs_writeHead

##jswrap_http_createServer

JsVar *jswrap_http_createServer(JsVar *callback)

The callback function supplied in the callback parameter is invoked when a new browse connection is received. The callback function has the form:

function(request, response)

Where request is the connection for the incoming data and response is the connection for the outgoing data. For example, we can write data to the "response" object and read data from the "request" object.

The return from the jswrap_http_createServer function is an instance of an httpSrv object.

##jswrap_http_get

JsVar *jswrap_http_get(JsVar *options, JsVar *callback)

Send an HTTP request to an HTTP server. The options variable defines the parameters of the connection and includes properties for host and port. The callback function is a function that will be invoked when a response is received. The signature of that functions is:

function(response)

Where response can be used to retrieve data and determine when the response connection is closed.

The jswrap_http_get returns an instance of an httpCRq objetct.

##jswrap_httpSRs_write

bool jswrap_httpSRs_write(JsVar *parent, JsVar *data)

##jswrap_httpSRs_end

void jswrap_httpSRs_end(JsVar *parent, JsVar *data)

##jswrap_httpSRs_writeHead

void jswrap_httpSRs_writeHead(JsVar *parent, int statusCode, JsVar *headers)

#End User network programming From a programmers perspective who is writing JS networking, there is a library exposed called net. I has the following exposed methods:

  • net.connect - Connect to a partner
  • net.createServer - Become a server

There also appears to be classes that leverage this library. These classes are Socket and Server.

##Socket ###Socket.available Socket.available()

Returns how many bytes are available to be read.

###Socket.end Close the socket.

###Socket.pipe Pipe the data to a stream.

###Socket.read Read some characters from the socket.

###Socket.write Write some data to the socket.

###Socket.close Event: Called when the connection closes.

###Socket.data Event: Called when data has been received and is available to be read.

###Socket.drain Event: Called when the data has been transmitted and new data can be sent.

##Server

#Adding a new network device To add a new network device, the device must initialize the Espruino environment by calling

networkCreate and then networkSet. Here is an example piece of code:

JsNetwork net;
networkCreate(&net, JSNETWORKTYPE_ESP8266_BOARD);
networkSet(&net);

In the network.c file there is a bootstrap mechanism in the function called networkGetFromVar which switches on the network type and calls a function called netSetCallbacks_<networkType>.

It appears that we have to supply the following functions

  • int net_<board>_accept(JsNetwork *net, int serverSckt)
  • int net_<board>_recv(JsNetwork *net, int sckt, void *buf, size_t len) - Return the number of bytes actually sent.
  • int net_<board>_send(JsNetwork *net, int sckt, const void *buf, size_t len)
  • void net_<board>_idle(JsNetwork *net)
  • bool net_<board>_checkError(JsNetwork *net)
  • int net_<board>_createSocket(JsNetwork *net, uint32_t host, unsigned short port)
  • void net_<board>_closeSocket(JsNetwork *net, int sckt)
  • void net_<board>_gethostbyname(JsNetwork *net, char *hostName, uint32_t *outIp)

#API Reference

####networkGetAddressAsString

#####Call type: JsVar *networkGetAddressAsString(unsigned char *ip, int nBytes, unsigned int base, char separator)

#####Description Return a JS variable that represents the IP address.

#####Parameters

  • ip - The IP address in memory.
  • nBytes - The size of the IP address (usually 4).
  • base - The base representation (usually 10 for decimal).
  • separator - The separator character (usually '.').

#####Returns A representation of the IP address.

####networkParseIPAddress

#####Call Type:

uint32_t networkParseIPAddress(const char *ip)

#####Description

Parse a string representation of an IP address and return an IP address.

#####Parameters

A string representation of an IP address.

#####Returns

A 4 byte IP address.

Add a custom footer
Add a custom sidebar
Clone this wiki locally