Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SIP008] Online config #89

Open
2 tasks done
CzBiX opened this issue Oct 30, 2017 · 54 comments
Open
2 tasks done

[SIP008] Online config #89

CzBiX opened this issue Oct 30, 2017 · 54 comments

Comments

@CzBiX
Copy link

CzBiX commented Oct 30, 2017

Motivation

When the server changes SS config(such as address, port, method, password), the client can auto-update config without user interaction.

Overview

  1. server provides the URL of the online config file.
  2. client fetch and save online config.
  3. client periodically checks for config updates.

Basic file format

Basically, we follows the original JSON config format of shadowsocks. The required fields are:

{
  "server": "198.199.101.152",
  "server_port": 8388,
  "password": "u1rRWTssNv0p",
  "method": "aes-256-cfb",
  "remarks": "Example 1"
}

To support multiple servers, put those server objects into one JSON array.

Extended file format

The JSON config can be extended with any client specific field. Any unsupported filed can be just ignored.

{
  // config file struct version
  "version": 1,
  "remark": "Some server",
  "servers": [
    {
      // config id, so the client can detect it's a new server or just changed password
      "id": "1a2b3c4d",
      "remarks": "Server A",
      // SS connect config, same as shadowsocks-libev
      "server": "example.com",
      "server_port": 1234,
      "password": "example",
      "method": "chacha20-ietf-poly1305",
      "plugin": "xxx",
      "plugin_opts": "xxxxx"
    },
    // another server config
    {
      "id": "1a2b3c4e",
      "remarks": "Server B",
      "server": "example.com",
      "server_port": 4321,
      "password": "example",
      "method": "chacha20-ietf-poly1305",
      "plugin": "xxx",
      "plugin_opts": "xxxxx"
    }
  ]
}

Remark

  1. The server SHOULD provide HTTPS to avoid MITM, and the client MUST NOT ignore cert error.
  2. The config URL MAY include query params to identify a user as authentication.
  3. The config URL MAY return HTTP redirects, the client SHOULD follow it at least once.
  4. The client MAY check updates on App start/open/connected, SHOULD provide a button to force a refresh.
  5. The server MAY omit some field in the config, except id, server, server_port, password, method in servers.

TODO

  • Improvement this spec document.
  • Implement in clients that have server manager feature(such as shadowsocks-windows, shadowsocks-android).
@Mygod
Copy link
Contributor

Mygod commented Nov 2, 2017

Where's authentication? This proposal will get GFW-ed within days. You just don't publish your server on the Internet.

@Mygod Mygod closed this as completed Nov 2, 2017
@CzBiX
Copy link
Author

CzBiX commented Nov 2, 2017

只要使用了 HTTPS 并且保证 URL 的安全,GFW 就无从得知包括 URL 在内的信息。
这就拥有了一套实现简单的防窃听及认证机制。

我提出这个提案的目的主要是为了解决朋友之间共享服务器时能便捷的更新代理信息。
就算是中心化的代理站,我也想不到这个会带来什么新的问题。

朋友之间没有必要把 URL 公布到网上,代理站需要 URL 中的信息标识用户,更加不会公布 URL。

@Mygod
Copy link
Contributor

Mygod commented Nov 2, 2017

So you're using a secret in URL to reveal the secret key. The second secret is authentication. And also by doing this you have effectively introduced a handshake for the protocol.

One could work around the handshake and design a totally profitable protocol out of this:

  1. Set up an HTTPS server behind CDN, possibly with per-user authentication; (one could hide such page behind a seemingly innocent blog site)
  2. Use techniques similar to meek to obfuscate and do authentication with the server;
  3. Server sets up a random key and listens at some random port (or even a random server if you have a lot of money) and return the key-port pair;
  4. Client connects using the random key-port pair.

@madeye
Copy link
Contributor

madeye commented Nov 2, 2017

I suggest to host a QR code on a secret HTTPS website. It should do the same trick.

@CzBiX
Copy link
Author

CzBiX commented Nov 2, 2017

@Mygod I still didn't get the point about add authentication over HTTPS, and a complicated way at that.
Keep URL secret should be enough. If it really needs, HTTP auth also works well. (URL like https://user:[email protected])

@Mygod
Copy link
Contributor

Mygod commented Nov 2, 2017

Your URL can't be a well-known fixed string. And therefore there will be a redundant second secret.

@Mygod
Copy link
Contributor

Mygod commented Nov 2, 2017

The point is that one can design a better protocol with a secret URL as a single entry point than forcing it into the current framework (i.e. as an SIP), not that per-user authentication is necessary.

@CzBiX
Copy link
Author

CzBiX commented Nov 2, 2017

@madeye So, I have to notify everyone about config changed. Users have to scan QR code many times to add servers, find out which one is the new server, which one should be removed.

Yeah, We need more manual action to live in the modern digital world.

I think my servers is not good and stable as you have. And some of my friends don't meet the tech requirements to across the wall.

Deeply hope similar feature implemented in the future.

@madeye
Copy link
Contributor

madeye commented Nov 3, 2017

@CzBiX I agree that this proposal would be useful in many scenarios. In fact, it's also easy to implement. I believe @Mygod can implement it in one line with Scala. 😄

However, to become a new feature, we need to consider more, for example, the authentication requirement here.

I think you can keep thinking about this proposal, polish your design and collect more feedback from the community. If so, I guess we would finally have a new SIP.

@madeye
Copy link
Contributor

madeye commented Nov 3, 2017

Reopen the issue to allow @CzBiX refine his proposal further.

@madeye madeye reopened this Nov 3, 2017
@CzBiX
Copy link
Author

CzBiX commented Nov 3, 2017

@madeye Thanks.

Sorry, maybe I don't know much about Internet security, I still can't understand why we need more authentications.
If I choose to share config via SS URI or QR code as before, it also is well-known info. What's the difference? Should we move password outside SS URI to force user input it?

In my mind, the client can use HTTPS cert to verify server's identity, and server identifies client use auth info exists in URL, the provider can choose which security level and the method they want:

  • Keep URL be secret, so it works just like the password. Of course, simple URL like /ss_config.json is equivalent to a simple password, and service provider should avoid it.
  • Use HTTP auth like https://user:[email protected]/config, or use token like https://example.com/config?token=xxxx.

Both of them easy to implement on server side, and easy to use because of just single URL on client side.
Yes, I want to keep SS be easy to configure and use as it always does.

@Mygod
Copy link
Contributor

Mygod commented Nov 3, 2017

Here's my proposed change if you really want this kind of thing.

Motivation

Current shadowsocks configuration involves many parameters. To add insult to injury, the introduction of plugins (#28) introduces even more configurations. As a result, new ss URLs are usually very bloated (#27).

All of this would be fine if the server never changes its configuration. However this is not likely to happen due to following reasons:

  1. Censors are constantly getting more powerful. As a result, one may want to use different configurations to obtain the best result.
  2. Even if the Shadowsocks protocol no longer changes, it's very likely that there will be changes to plugins.

Whenever the server updates its configuration, due to the handshake-less nature of the protocol, every client has to update their configuration accordingly. This can be a tedious process if the number of clients is large. Ideally we want to use only one single pre-shared key to authenticate valid users.

Protocol

Configurations via HTTPS

First, the config file is published on the Internet following one of these two URL schemes with an optional front domain parameter as the fragment: (TODO: which one?)

a. https://<domain>/<hard-to-predict-path>.ssconf[#[front-domain.example][;remark]]
b. ssconf://<domain>/<hard-to-predict-path>[#[front-domain.example][;remark]]

The web server hosting this ssconf page could be:

  1. The same server as shadowsocks server or
  2. A different server, or hosted on a trusted provider such as github.io, heroku, etc.

It's recommended to put the web server behind a trusted CDN, preferably the ones supporting domain fronting.

Upon GET request, the web server validates whether the request is performed using HTTPS and returns an ordinary shadowsocks json configuration in that case. Example:

{
  "server": "example.com",
  "server_port": 1234,
  "password": "example",
  "method": "chacha20-ietf-poly1305",
  "plugin": "xxx",
  "plugin_opts": "xxxxx"
}

The remark field returned from server will be ignored and the remark in URL fragment will be used instead. Multiple configurations should be put at different URLs.

Client Behavior

  1. The client should take a single HTTPS URL as the entrypoint;
  2. If front domain was not supplied, the client should use the domain in the URL as front domain;
  3. Client resolves the supplied front domain in the DNS query and send the HTTPS GET request to the resolved address with front domain as SNI;
  4. Client obtains the server's certificate;
  5. Upon first connection, the client should validate the certificate against the REAL domain instead of front domain:
    a. If it's trusted, display a warning dialog with the server's certificate and asks the user to confirm the certificate;
    b. If it's not, display an error dialog with the server's certificate. Provide a hidden option (as in Chrome) for the user to confirm the certificate;
  6. Otherwise, the client should validate whether the certificate has changed:
    a. If it changed, go to 5b;
    b. If not, go to 8;
  7. If user confirms the new certificate, store the new certificate, otherwise go to 9;
  8. Finish the HTTPS request and store the configuration in the response;
  9. If there's a valid configuration in the store, use that configuration to connect, otherwise terminate.

Note that in order to guard against MitM attacks, the client now should be stateful to prevent country-scale certificate forgeries. Here's what a console ss-local would look like:

$ ss-local [--storage /path/to/protected/storage] -l 1080 --fast-open [--other-client-params] <URL>
Certificate Information for <real-domain>
  Issued to:  <real-domain>
  Issued by:  DigiCert SHA2 Extended Validation Server CA
  Other information...
This certificate looks good. Proceed to connect? [y]/n
 INFO: using tcp fast open
 INFO: initializing ciphers... chacha20-ietf-poly1305
 INFO: tcp port reuse enabled
 INFO: udprelay enabled
 INFO: udp port reuse enabled
 INFO: listening at 127.0.0.1:1080

Remark

  1. URL path MUST be hard to predict (have enough entropy) but need not hide the underlying configurations. It could be something like /that-outcrop-infuriates-my-bud-my-shadowsocks-config-with-kcptun-as-plugin-japan-123.4.5.67-password-is-hello-world or /A9FEF2E0C966D101B1C5E0CCB49DCC8BE251E1E6DB53F04E11EE54C2593E6BAD36184A8684B9CE1639D7BF09FEA50CB7047536C7B252FFDA60FEC61ACE6BF57D.
  2. It's not recommended to use query params and/or HTTP authentication to do authentication. Use url path instead. For example one could use /config/<secret-tag>. No matter which one you choose, the server MUST return HTTP 404 if authentication fails instead of the traditional 401 or 403.
  3. It's recommended to host a seemingly innocent website for other requests, such as a blog.
  4. Clients should treat redirects (HTTP 3xx) as errors. due to possible certificate issues. Web servers should handle URL changes internally.
  5. Upon configuration updates, it shouldn't introduce changes that would result in IP blocks for old clients. For example, whenever password and/or method changes, the port should change as well if possible.
  6. It's not recommended to use the same server as the web server without CDN. It's harder to set up, it makes the server more vulnerable and easier to fingerprint, but it shouldn't be too problematic.
  7. Password in online configurations should still have as much entropy if not more since the user won't ever need to enter them manually.
  8. Which URL scheme should we use? Both of them should work fine for Android. The first one looks better but I suspect only the second one can be imported with one click under browsers like Chrome.

@ghost
Copy link

ghost commented Nov 4, 2017

I think it's better to use a separated program / script to handle this logic (like ss-manager, but for client side), and keep ss-local simple.

@Mygod
Copy link
Contributor

Mygod commented Nov 8, 2017

It's just a demo of usage, and also how complicated it is to implement securely. (it's nothing you can do with one line of scala)

@WordlessEcho
Copy link

WordlessEcho commented Nov 14, 2018

Deleted

@fortuna
Copy link
Contributor

fortuna commented Nov 29, 2018

I like the idea of an online config. It can be blocked the same way the Shadowsocks server is. However, it's something you access rarely, so it will draw a lot less attention and it will be a lot harder to find and block.

I have some thoughts about practicality of the idea.

Colocation

Quite often the most convenient is to have the config live where the server is, but that means the config will be blocked if the server is. But there are solutions:

  • Multiple IPs: DigitalOcean givies you a free floating IP per server, so you can have one IP for Shadowsocks and one IP for the webserver with the config. Other cloud providers have similar features, though it may not be free. This solution is convenient to those that don't have a domain name.
  • Set up a CDN: You can configure Cloudflare for free as a reverse proxy for your config. Besides giving you an alternate address, it allows for domain fronting. However I believe it requires the admin to provide a domain name.

One issue we ran into in Outline is that users often don't have a domain name. Which means they can't get a TLS certificate from Let's Encrypt. Because the certificates are self-signed, we need a certificate fingerprint to validate them. The configuration URL should provide a way to specify the TLS certificate fingerprint, so clients can validate certificates when content is served from an IP address.

Publishing

Alternatively, you may publish the config somewhere else. You'll want to automate that, which means generating some API key and wiring your code properly, which is often not quite straightforward if you've never done it before.

Notice that you don't really need the fetch URL to be private. The config page can be encrypted and published publicly anywhere. It may even use HTTP, but you will want an AEAD cipher to validate authenticity. The config URL can have the secret key used to decrypt the content by the client. Ideally in the fragment, to make sure it's never sent to the host server.

Publishing requires you to trust the host, which is not true for many people, unless you use the encrypted approach above.

Some options for publishing:

  • AWS or some cloud storage that is not blocked. This requires setting up cloud storage, which is complicated, and often requires payment. If you are using a secret URL, make sure to disable file listing!
  • Publish to Github. This is a free solution, but requires setting up a github account and repository. You could use a Gist instead, which just requires an account. You can list the Github contents, so it requires the encrypted content approach.
  • Google Drive or other drivers. Requires having a Google account and uploading the config. It's blocked in China though.
  • Google Sites, Blogger. An alternative to Drive, but also blocked in China.

URL Shortener

An alternative to publishing a config is to use an editable URL Shortener. In that case it could redirect directly to the ss:// link.

As in the case of publishing, you need to trust the URL Shortener.

Backward compatibility

When you update the config, make sure the old configuration keeps running until all your clients have migrated to the new config. Otherwise you will break them.

@qiuyuzhou
Copy link

qiuyuzhou commented Sep 8, 2019

A lots of people open issues for implement of this feature in Shadowsocks-NG. Recently I consider it again.

  1. Could return multiple servers in one configure file. The service provider could provide a group servers updating in one endpoint.
  2. After get the new online configure file, the client can delete the old profiles which has been removed from the online file.
  3. Add, update or delete by identify the profile by the host:port pair.
  4. Serve in a hard-to-predict-path as @Mygod suggested.
  5. Only served by https (Not http) in order to keep simple and secure.
  6. @fortuna has mention an AEAD cipher approach to serve the config file by http. But it's complex to implement it. Should be easy to implement it by all client and server.
  7. Only in json format.

To be simple to edit, I show the example in yaml format:

# Identify the group of servers.
# Can be used by client to track the servers from same source.
# When update, the client can use the id to deleted the old servers 
# which has been removed from the server list.
id: com.xxx.foo/bar
# Use the SIP002 format URL instead of a new spec. Keep it simple.
servers:
  - ss://[email protected]:8888#Example1
  - ss://[email protected]:8888/?plugin=obfs-local%3Bobfs%3Dhttp#Example2

@fortuna
Copy link
Contributor

fortuna commented Nov 11, 2019

  1. @fortuna has mention an AEAD cipher approach to serve the config file by http. But it's complex to implement it. Should be easy to implement it by all client and server.

The client could leverage the same crypto that the Shadowsocks protocol uses, so it's just a matter of reusing libraries. In Outline, we could use our ShadowsocksReader.

@fortuna
Copy link
Contributor

fortuna commented Nov 11, 2019

For the config URL, we should consider not only a front domain, but also a domain to resolve and get the IP address from. For example, we can resolve cloudflare.com to get IPs for any cloudflare website.

These are the things we may need in what we give the client:

  • Domain to use in the SNI and as the default for IP resolution and Host header.
  • Secret HTTP path to fetch
  • Optional: Domain to resolve to get the IP address (or a hard-coded IP address directly)
  • Optional: Domain to use in the HTTP Host header.

If we want to support self-signed certificates for users without a domain name:

  • IP address of the server
  • Fingerprint of the certificate
  • Secret HTTP path to fetch

@ghost
Copy link

ghost commented Jan 25, 2020

I'd like see it become standard.

Shadowsocks-windows has another experiment few years ago:
https://github.com/shadowsocks/shadowsocks-windows/tree/with_online_config

My concern:

  1. If online config use ssconf://, we can register protocol handler for it. (But we haven't do it for ss:// yet)
  2. On-wire format should be extensible, so other implemention such as Outline-server can send extra config they (PAC URL, DNS server, etc.) in it.
  3. Self-signed cert would introduce extra complexity when implemention:
  • Can we update it via online config?
  • Should we validate it? How to store public key?
  1. For authentication, how about TLS Client Authentication?

@madeye
Copy link
Contributor

madeye commented Jan 25, 2020

We're finalizing the online config support with an extendable JSON format. Here's an example: https://gist.github.com/madeye/8451355b1baba312dae78a6c8ca4d1a1

Generally, we don't limit any custom fields in the JSON config file. The only required fields are:

  {
    "server": "198.199.101.152",
    "server_port": 8388,
    "password": "u1rRWTssNv0p",
    "method": "aes-256-cfb",
    "remarks": "Example 1"
  }

@ghost
Copy link

ghost commented Jan 25, 2020

More about authentication

  1. POST a form(or json or other stuff) to server is traditional but still useful.
  2. Custom HTTP header (Like X-Token) is a popular option.
  3. Combine 1 and 2, and add some web technology, we can get this:
    1. Client POST user name and password to login URL
    2. Server allocate a token (generate by JWT?) to client
    3. Client GET online config URL with given token in custom header
    4. Server check the token and send the config

Option 1 is easy, but require POST
Option 2 only need GET and also easy to implement
Option 3 is hard to implement but easy to manage

@madeye
Copy link
Contributor

madeye commented Jan 25, 2020

GET is good enough, as we're forcing HTTPS URL for any online config.

You can put any restful token in your URL for authentication.

@madeye madeye changed the title [SIP] Online config [SIP008] Online config Feb 2, 2020
@madeye
Copy link
Contributor

madeye commented Feb 2, 2020

The latest shadowsocks-android has already supported this SIP: https://github.com/shadowsocks/shadowsocks-android/releases/tag/v5.0.3

@database64128
Copy link
Contributor

@madeye maybe it’s time to formalize the format by putting it on our website.

@madeye
Copy link
Contributor

madeye commented Nov 9, 2020

@database64128 PR is welcome. 😄

@database64128
Copy link
Contributor

@madeye I wish I could, but the spec folder points to the wiki repository, which I can't contribute to without push permissions. 😅

@madeye
Copy link
Contributor

madeye commented Nov 9, 2020

Okay, I forgot you need to add pages here: https://github.com/shadowsocks/shadowsocks-org/wiki

@database64128
Copy link
Contributor

Thanks y'all for all the brilliant ideas! I just published the draft document of SIP008 Online Configuration Delivery to the wiki. Please review and share your thoughts. Once everything is cleared up, I'll publish the standard document to our website. 😄

https://github.com/shadowsocks/shadowsocks-org/wiki/SIP008-Online-Configuration-Delivery

@zonyitoo
Copy link
Contributor

zonyitoo commented Nov 10, 2020

// Optional fields for data usage:
"bytesUsed": 274877906944,
"bytesRemaining": 824633720832

Why do these two fields have different naming style?

@database64128
Copy link
Contributor

@zonyitoo You got me! 😅 Changed to follow the naming convention of snake_case.

@alalamav
Copy link
Contributor

We have started prototyping support for online config in the Outline clients. I agree with @database64128 in that standarizing this proposal will strengthen the Shadowsocks ecosytem. We would like for Outline clients to remain compatible with standard Shadowsocks servers. Here are some thoughts and needs that we have found so far:

URL protocol

I believe that registering a custom protocol provides a better user experience than manually adding the URL to the app. Additionally, Shadowsocks users are already familiar with the ss protocol - ssconf would use this fact as a building block.

I realize that URL interception is not possible in every platform, so clients should support manual import. The https protocol cannot be registered since we don't know the hostname in advance.

URL format

We are exploring how to colocate an access service (online config server) to Outline servers. The current URL format works well for servers that have domain names and the corresponding (CA signed) certificates. Most Outline servers don't have domain names and rely on self-signed certificates for accessing the management API. In order to prevent MITM attacks, we communicate the certificate fingerprint out-of-band to the client, pin it, and validate upong connecting. This solution requires access to native networking APIs, since browsers (webviews) cannot easily pin custom certificates, but it has worked well for Outline servers, so we would like to extend it to online config.

In order communicate the certificate fingerprint to the client, we may want to define a more flexible URL format. We propose to encode arbitrary options in the URL tag, much like SIP002 plugin options:

ssconf://<domain>[:port]/<hard-to-predict-path>[#param0=val0;param1=val1...;]

The domain front can be encoded in the tag, along with the remark: #domain_front=example.com;remark=ss%20server;

Self-signed certificates

With the new URL format we may avoid displaying a user prompt to confirm untrusted cetificates, as long as the certificate fingerprint is present. The client should terminate untrusted connections that don't provide a valid certificate fingerprint.

HTTP redirects

The proposal states that, "Clients should treat redirects (HTTP 3xx) as errors due to possible certificate issues. Web servers should handle URL changes internally."

HTTP redirects can be a way to migrate online configs. The server may redirect clients to an ssconf URL in order to mitigate potential certificate issues. The client should persist the online config URL for permanent redirects.

File format

I don't feel strongly about the JSON file formats proposed on this thread. As long as we are able to support mulitple servers, are backwards compatible with Shadowsocks naming conventions, and can arbitarily extend individual configurations. The more important aspect is that the format is standardized.

I'm interested in hearing your thoughts and in finding common ground to move this proposal forward. Thanks!

@fortuna
Copy link
Contributor

fortuna commented Nov 12, 2020

URL Redirects

I would like to emphasize @alalamav 's case to support HTTP redirects. We ought to support that because it's easy and brings multiple benefits:

  • We can use a redirect to migrate a user to a new config, in a different location.
  • We can implement a mechanism where a shared URL can do a permanent redirect to a new user-specific URL. From that point on, the user will only use the user URL and forget the shared URL. This way we can share one URL, but each user will get their own different config. This enables easy safe sharing at scale. We intend to implement this mechanism on Outline Servers.
  • We can redirect to a ss:// url and bypass format issues.

This is how it can work:

  • Replace ssconf:// with https:// and fetch the content. Use the parameters to validate the response.
  • If it's a 2xx response, use the config.
  • If it's a redirect, repeat with the new location

If the first redirect from the saved URL is a permanent redirect, we should replace the saved URL with the new one, to allow for migrations. We should only do this once we confirm the new config works.

A HTTPS url should be able to redirect to a ssconf:// so we can redirect to a location with a self-signed certificate.
A HTTPS url should also be able to redirect to a ss:// location. In which case you can stop and use that config (same as if you entered the ss:// link in the first place).

Config Refresh

We should specify when the client should fetch the config, so the server can migrate a user (for example, to rotate IPs) to a new config seamlessly, without interrupting the existing connections. Note that seamless transitions require supporting an old config until a client migrates to a new config.

A client app should fetch the config every time a client hits "Connect", and then again before the config expires. That means we need a way to communicate a config TTL or expiration time to the client somehow. For example, if you have a TTL of 2 hours, that means the server manager can stop supporting an old config 2 hours after a new config is made available. I recommend a TTL instead of an expiration time, because you can keep using the same value without updating.

A TTL can be used to stop serving idle configs and save resources.

A TTL can be used to force the user to refresh their config in a short period and the server can generate a new one whenever the TTL expires. This provides forward secrecy with Shadowsocks!

This TTL can be in the config somewhere or in a HTTP header. I'm still unsure where.

File format

The proposed file format does not differentiate fields that are required from fields that are optional. This will prevent future extensibility. I need to know how to handle field I don't know of. Should I ignore, or should I abort? ID and remarks are optional. method and password are not. Plugin is also required. Outline doesn't support plugins, so we need to know if there's a plugin field we should reject the config. But Outline needs to know that the plugin field exists, and is required.

Perhaps we have a structure like:

{
    "version": 1,
    "servers": [
        {
            "id": "27b8a625-4f4b-4428-9f0f-8a2317db7c79",
            "remarks": "Name of the server",
            "proxy_config": {
                "server": "example.com",
                "server_port": 8388,
                "password": "example",
                "method": "chacha20-ietf-poly1305",
                "plugin": "xxx",
                "plugin_opts": "xxxxx"
            }
        },
        {
            "remarks": "Name of the server",
            "proxy_config": {
                "server": "example.com",
                "server_port": 8388,
                "password": "example",
                "method": "chacha20-ietf-poly1305",
                "plugin": "xxx",
                "plugin_opts": "xxxxx"
            }
        }
    ]
}

And we define that every field under proxy_config must be processed. If there's an unknown field, the server should be discarded.

@database64128
Copy link
Contributor

  • Under the draft document, clients are supposed to act as an ordinary HTTP client, which should follow HTTP redirects.
  • On iOS, a URL scheme can only be associated with one app. On Windows, the same limitation applies to Win32 apps. It's quite common for users to have multiple anti-censorship apps installed. I'm skeptical that registering the URL scheme actually improves the user experience on these platforms. As far as I know, when I open Outline Client on Windows, it overwrites the ss:// association registered by shadowsocks-windows. Note that for shadowsocks-windows, ss:// association is opt-in.
  • I also have mixed feelings about the proposal to add a mechanism to allow self-signed certificates. It's certainly useful given how Outline servers are deployed. But it's only secure when security protocols are strictly followed. We must be careful when standardizing the use of self-signed certificates, as we don't want to encourage it. But if one has to do it, one must do it right.
  • I have plans for an SIP008 extension, which gives authors of other anti-censorship protocols an opportunity to register their protocol and reserve their custom fields.
  • @fortuna @alalamav What you proposed are all valid points. But many of us just want a simple online configuration delivery protocol that works. The draft document of SIP008 fits exactly into that need: simple yet extensible. Your might want to draft a new SIP010 for your proposals. 😅

@fortuna
Copy link
Contributor

fortuna commented Nov 13, 2020

@database64128 We'll probably support the standard specified here, but we'll go further and extend it to support the advanced cases we need (self-signed certs, redirects, config rotation). If we can demonstrate it works well, then we can present back to the community for standardization.

On this proposed standard, It would be great to follow the caching directives whenever possible. In particular, The Cache-Control header with max-age is quite useful. There's a nice tutorial on HTTP caching at https://www.mnot.net/cache_docs/

@fortuna
Copy link
Contributor

fortuna commented Nov 17, 2020

@studentmain @database64128 @madeye What's the retrieval logic for shadowsocks-android and shadowsocks-windows?

  • When do they retrieve the config? Is it every time the user clicks connect? Is it only when adding the config?
  • Do they follow redirects?

@database64128
Copy link
Contributor

In the upcoming shadowsocks-windows v5, users will be able to specify an interval for automatic retrieval. Redirects are followed.

@zonyitoo
Copy link
Contributor

zonyitoo commented Nov 24, 2020

If a server does not use a plugin, the plugin and plugin_opts should be empty or excluded from the server object.

"plugin" and "plugin_opts" shouldn't be mandatory, which will results in configuration files to have useless informations (tons of keys with empty values).

@database64128
Copy link
Contributor

"plugin" and "plugin_opts" shouldn't be mandatory, which will results in configuration files to have useless informations (tons of keys with empty values).

The plugin and plugin_opts fields are not mandatory. But since empty strings are a clear representation of not using plugins, the standard document is being explicit and permissive about allowing empty strings to maximize compatibility.

@beesuns

This comment has been minimized.

@fortuna
Copy link
Contributor

fortuna commented Apr 21, 2021

@database64128, @madeye , I'm trying to understand what what ss-windows and ss-android clients currently support to see if Outline can generate links that will work with your clients. Would you mind answering for your client:

  • Does it support ssconf:// urls?
  • What happens if the certificate is self-signed?
  • Do you follow redirects? (it seems windows is yes)
    • Do you do anything special about permanent redirects (like replacing the url you will fetch in the future)?
    • Do you support redirect to a ss:// link?
  • Do you fetch the config whenever the user starts a session? When do you fetch the config?

/cc @JonathanDCohen @alalamav

@database64128
Copy link
Contributor

Does it support ssconf:// urls?

It currently doesn't. But we do have plans to add support in an upcoming version.

What happens if the certificate is self-signed?

The certificate gets rejected and the fetch fails.

Do you follow redirects? (it seems windows is yes)

Yes.

Do you do anything special about permanent redirects (like replacing the url you will fetch in the future)?

No. It shouldn't be difficult to implement, but I'm not sure why we would need that. It doesn't sound like a reliable way to communicate the online config URL change.

Do you support redirect to a ss:// link?

No. Again, it's easy to implement, but wouldn't it be better to just stick to the spec and return an online config JSON?

Do you fetch the config whenever the user starts a session? When do you fetch the config?

Fetch on app startup.

If you are looking for a reference implementation of SIP008-compliant online config provider, take a look at database64128/shadowsocks-uri-generator. It's a set of tools that manage federated Shadowsocks servers, with support for Outline API. Server config is delivered per the SIP008 spec. Online config URL delivery and user interactions are enabled by the Telegram bot.

@database64128
Copy link
Contributor

Since I no longer have control over the SIP008 spec, I'm starting Open Online Config for online config protocols and API specs. Everyone is welcome to review the draft of Open Online Config 1 that supersedes SIP008.

@septs
Copy link

septs commented Sep 24, 2023

SIP008 standardizes unspecified SIP008 format MIME types

I suggest, use text/sip008+json

If needed, suggest application for a new media type, see https://www.iana.org/form/media-types

HTTP Example

Request:

GET /to/path HTTP/1.1
Accept: text/sip008+json, text/uri-list

Response:

HTTP/1.1 200 OK
Content-Type: text/sip008+json

References

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests