All installations are currently being built from fresh sources for reasons of development and keeping up to date with core, evolving features. As such, installations make take extensive time to build, especially on slower machines.
sudo apt-get install -y gitgit clone https://github.com/alecmuffett/eotk.gitcd eotk./opt.d/build-raspbian-stretch.sh
Install a ubuntu-20.04.2-live-server-amd64.iso server instance; and then:
git clone https://github.com/alecmuffett/eotk.gitcd eotk./opt.d/build-ubuntu-20.04.sh
Install a ubuntu-18.04.2-live-server-amd64.iso server instance; and then:
git clone https://github.com/alecmuffett/eotk.gitcd eotk./opt.d/build-ubuntu-18.04.sh
Install a minimal server and then:
sudo yum -y install gitgit clone https://github.com/alecmuffett/eotk.gitcd eotk./opt.d/build-centos-8.2.2004.sh
Install Homebrew; and then:
git clone https://github.com/alecmuffett/eotk.gitcd eotk./opt.d/build-macos-mojave.sh
Install a base server and then:
pkg install gitgit clone https://github.com/alecmuffett/eotk.gitcd eotk./opt.d/build-freebsd-12.1.sh
EOTK requires recent tor and also recent openresty with the
ngx_http_substitutions_filter_module NGINX module enabled. They
simply need to be accessible in $PATH. The above "build" scripts
simply supply that.
After installation, you can do:
./eotk config demo.d/wikipedia.tconf- or:
./eotk config demo.d/wikipedia-v3.tconf
- or:
./eotk start wikipedia./eotk maps -a# and connect to one of the onions you've created
Be aware that you will suffer from HTTPS certificate errors until you buy a HTTPS certificate.
When connecting to the resulting onions over HTTP/SSL, you will be using wildcard self-signed SSL certificates - you will encounter many "broken links" which are due to the SSL certificate not being valid.
This is expected and proper behaviour; there are currently three ways to address this.
The best solution for development purposes is to install mkcert
onto the machine which will be running
EOTK and
configure your own personal Certificate Authority for the certificates
that you will need.
You can then add set ssl_mkcert 1 to configurations, and your
mkcert root certificate will be used to sign the resulting onion
certificates. You can install that certificate into your local copy
of Tor Browser;
of course it will not work for anyone else.
The old solution was/is much more manual: EOTK will use OpenSSL to create certificates, and then, for any onion - eg: www.a2s3c4d5e6f7g8h9.onion - EOTK provides a fixed url:
https://www.a2s3c4d5e6f7g8h9.onion/hello-onion/
...which (/hello-onion/) is internally served by the NGINX proxy and
provides a stable, fixed URL for SSL certificate acceptance; inside
TorBrowser another effective solution is to open all the broken links,
images and resources "in a new Tab" and accept the certificate there.
In production, of course, one would expect to use an SSL EV certificate to provide identity and assurance to an onion site, rendering these issues moot.
See below.
If you choose to buy an Onion HTTPS certificate from (e.g.) HARICA, what will happen, and what will you need to do?
I chose to buy:
- a Server Certificate
- with Domain-Level (DV) Trust
- with a reasonable duration
- using the in-browser generated CSR
- using the ECDSA algorithm at 256 bits
- important: remember the password!
- important: download the private key!
The HARICA website provides an in-browser method of generating a CSR,
if you are not already familiar. Make sure to generate a good
passphrase, and remember it, because you will need it soon.
Also: make sure to download the privateKey.pem file that
is offered, and keep it in a safe place.
If you manually create the CSR on the server, you'll use the 'onionaddress.key' file generated by openssl.
For example: HARICA will tell you that you need to post a secret key at a particular URL on your onion site; the message will read something like:
Place the file FILENAME to http://ONIONADDRESS.onion/.well-known/pki-validation/
...and they will offer you a file to download.
Download this file, and open it with a text editor; the content will be a long secret string, like THISISAREALLYLONGHEXADECIMALSECRET
Add a line to your EOTK configuration, substituting the values where necessary:
set ssl_proof_csv /.well-known/pki-validation/FILENAME,THISISAREALLYLONGHEXADECIMALSECRET
Then do something like:
eotk config projectname.conf && eotk nxreload projectname
...to install the URL handlers.
HARICA has a process of validation which involves generating an onion-csr. You won't need to add anything to your configuration.
You can put multiple path,value strings into ssl_proof_csv, space-separated;
use trailing backslashes to put entries onto separate lines:
set ssl_proof_csv \
/.well-known/pki-validation/key1,value1 \
/.well-known/pki-validation/key2,value2 \
/.well-known/pki-validation/key3,value3
The ssl_proof_csv hack works if all the proof URLs are
different; but if Digicert (or whomever) were to give you the
same pathname (e.g. /.well-known/pki-validation/fileauth.txt)
for all of the onions, what do you do?
Answer: you use "splicing". If you have onion addresses named
xxxxxxxxxxxxxxxx and yyyyyyyyyyyyyyyy, then you can create files:
templates.d/nginx-site-xxxxxxxxxxxxxxxx.onion.conftemplates.d/nginx-site-yyyyyyyyyyyyyyyy.onion.conf
...and into each put something similar to the following incantation; customise as necessary:
location = "/.well-known/pki-validation/fileauth.txt" {
return 200 "RESPECTIVE-PROOF-STRING-GOES-HERE";
}
...then when you next eotk config and eotk nxreload, that code
should be spliced into the correct configuration for each onion.
For each certificate, HARICA will offer you several files to download;
download the "PEM Bundle" file and copy it to your EOTK server.
Also: copy the privateKey.pem file (mentioned above) to the EOTK server.
Next, change Directory into ~/eotk/projects.d/PROJECTNAME.d/ssl.d;
you should see your development certificates, which will look like:
$ ls
ONIONADDRESS.onion.cert
ONIONADDRESS.onion.pem
There are two steps to installation:
Step 1: copy the PEM Bundle file from HARICA, on top of ONIONADDRESS.onion.cert
Step 2: unlock and extract (or rename) the private key, by doing:
openssl ec -in privateKey.pem -out ONIONADDRESS.onion.pem
...and typing in the password that you chose during the CSR setup, earlier;
if you chose to use RSA as the algorithm, you will need to use
openssl rsa ... instead.
If you manually created the CSR, then rename the 'onionaddress.key' file the CSR generated to 'onionaddress.onion.pem'.
Then: change directory back to the EOTK directory,
and do eotk nxreload projectname, and test it.
Once you have installed EOTK (below) and configured and tested it for your project, run:
./eotk make-scripts
This will create two files:
./eotk-init.sh- for installing on your system as a startup script./eotk-housekeeping.sh- for cronjob log rotation and other cleanup work
Please read the individual files for installation instructions; local setup is intended to be pretty simple, let me know if anything is confusing.
Okay, there are two ways to create your own project:
Create a config file with a .tconf suffix - we'll pretend it's
foo.tconf - and use this kind of syntax:
set project myproject
hardmap %NEW_V3_ONION% foo.com
hardmap %NEW_V3_ONION% foo.co.uk
hardmap %NEW_V3_ONION% foo.de
...and then run
eotk config foo.tconf
...which will create the onions for you and will populate a foo.conf
for you, and it will then configure EOTK from that. You should
probably delete foo.tconf afterwards, since forcibly reusing it
would trash your existing onions.
- Do
eotk genkey- it will print the name of the onion it generates- Do this as many times as you wish/need.
- Alternately get a tool like
scallion,shallot, oreschalotand use that to "mine" a desirable onion address.- https://github.com/katmagic/Shallot - in C, for CPUs
- Seems okay on Linux, not sure about other platforms
- https://github.com/lachesis/scallion - in C#, for CPUs & GPUs (GPU == very fast)
- Advertised as working on Windows, Linux; works well on OSX under "Mono"
- https://github.com/ReclaimYourPrivacy/eschalot - in C, for CPUs
- Works well under Linux, and BSD systems. Uses multiple threads. Can use a wordlist to generate onion addresses
- Be sure to store your mined private keys in
secrets.dwith a filename likea2s3c4d5e6f7g8h9.keywherea2s3c4d5e6f7g8h9is the corresponding onion address.
- https://github.com/katmagic/Shallot - in C, for CPUs
- Create a config file with a
.confsuffix - we'll pretend it'sfoo.conf- and use this kind of syntax, substitutinga2s3c4d5e6f7g8h9for the onion address that you generated.
set project myproject
hardmap secrets.d/a2s3c4d5e6f7g8h9.key foo.com
...and then (IMPORTANT) run:
eotk config foo.conf
...which will configure EOTK.
Like this:
eotk start myproject
When you are setting up the mappings in a config file, you may have to accomodate "subdomains"; the general form of a internet hostname is like this:
hostname.domain.tld# like: www.facebook.com or www.gov.uk- or:
hostname.domain.sld.tld# like: www.amazon.co.uk
- or:
hostname.subdom.domain.tld# like: www.prod.facebook.comhostname.subsubdom.subdom.domain.tld# cdn.lhr.eu.foo.nethostname.subsubsubdom.subsubdom.subdom.domain.tld# ...
...and so on, where:
- tld = top level domain
- sld = second level domain
- domain = generally the name of the organisation you are interested in
- subdomain = some kind of internal structure
- hostname = actual computer, or equivalent
When you are setting up mappings, generally the rules are:
- you will map one domain per onion
- you will ignore all hostnames
- you will append all possible subdomain stems
So if your browser tells you that you are fetching content from
cdn7.dublin.ireland.europe.foo.co.jp, you should add a line like:
hardmap %NEW_V3_ONION% foo.co.jp europe ireland.europe dublin.ireland.europe
...and EOTK should do the rest. All this is necessary purely for correctness of the self-signed SSL-Certificates - which are going to be weird, anyway - and the rest of the HTML-rewriting code in EOTK will be blind to subdomains.
Subdomains are supported like this, for dev as an example:
set project myproject
hardmap secrets.d/a2s3c4d5e6f7g8h9.key foo.com dev
...and if you have multiple subdomains:
hardmap secrets.d/a2s3c4d5e6f7g8h9.key foo.com dev blogs dev.blogs [...]
Example:
www.foo.com.auwww.syd.foo.com.auwww.per.foo.com.au,www.cdn.foo.netwww.foo.aws.amazon.com- ...
Put them all in the same project as separate mappings, remembering to avoid the actual "hostnames" as described above:
set project fooproj
hardmap %NEW_V3_ONION% foo.com.au syd per
hardmap %NEW_V3_ONION% foo.net cdn
hardmap %NEW_V3_ONION% foo.aws.amazon.com
Onion mapping/translations will be applied for all sites in the same project.
The logs for any given project will reside in
projects.d/<PROJECTNAME>.d/logs.d/
If something is problematic, first try:
git pulland...eotk config <filename>.confagain, and then...eotk bounce -a
This is probably an SSL/HTTPS thing.
Because of the nature of SSL self-signed certificates, you have to manually accept the certificate of each and every site for which a certificate has been created. See the second of the YouTube videos for some mention of this.
In short: this is normal and expected behaviour. You can temporarily fix this by:
- right-clicking on the image for
Open In New Tab, and accepting the certificate - or using
Inspect Element > Networkto find broken resources, and doing the same - or - if you know the list of domains in advance - visiting the
/hello-onion/URL for each of them, in advance, to accept certificates.
If you get an official SSL certificate for your onion site then the problem will vanish. Until then, I am afraid that you will be stuck playing certificate "whack-a-mole".
Generally this means that NGINX cannot connect to the remote website, which usually happens because:
- the site name in the config file, is wrong
- the nginx daemon tries to do a DNS resolution, which fails
Check the NGINX logfiles in the directory cited above, for confirmation.
If DNS resolution is failing, PROBABLY the cause is probably lack
of access to Google DNS / 8.8.8.8; therefore in your config file
you should add a line like this - to use localhost as an example:
set nginx_resolver 127.0.0.1
...and then do:
eotk stop -a
eotk config filename.conf
eotk start -a
If you need a local DNS resolver, I recommend dnsmasq.
If your onion project has just started, it can take up to a few
minutes to connect for the first time; also sometimes TorBrowser
caches stale descriptors for older onions. Try restarting TorBrowser
(or use the New Identity menu item) and have a cup of tea. If it
persists, check the logfiles.
Is the clock/time of day correct on all your machines? Are you running NTP? We are not sure but having an incorrect clock may be a contributory factor to this issue.
NEW FOR 2019: OnionBalance as-of June 2019 is an flaky piece of software which is hard to run on modern Linux because an stale python crypto library; more than 90% of Onion sites will not practically need it - or, not initially, anyway - so I am deprecating OnionBalance in EOTK until it is majorly overhauled and also supports v3 onion addressing.
So, I recommend people to avoid OnionBalance and use hardmap (local) for the moment, until Tor fix it. Consider OB if and only if your system is under sustained high bandwidth and strongly demonstrates extended choking on throughput.
These videos are instructive, but dated.
Commands have changed - but not very much - but please check the documentation rather than trust the videos; consider the videos to be advisory rather than correct.
