Internal Analysis Robot

[Mischback]

Idea

Have the (generated) website placed in a container (e.g. Docker) and let it be analyzed by a robot / spider.

Check every document and track the following information:

• internal links incoming (must be derived from analyzing other documents!)
• internal links outgoing (tracked by document, including counts, disregarding #targets)
• external links outgoing (tracked by URI, including counts, disregarding #targets)

Implementation in Python, most-likely multithreaded with a configurable number of worker threads beside the main management thread.