README

A collection of useful scripts and snippets by Akkana,
in various languges (but mostly Python).

All licensed under the GNU Public License v.2 or, at your option,
any later GPL version -- unless otherwise specified in comments
at the beginning of the script (there are a few scripts forked from
projects that use a different license.)

----------------
Subdirectories:
----------------

astro:
    Astronomy-related scripts

ebooks:
    Module of utilities useful with EPUB ebooks and Kobo ebook readers.

gimpdev:
    Helper scripts for working on the development version of GIMP.

[jsjupiter moved to my webapps repository]

mapping:
    Scripts related to mapping / GIS.

mapping/election2016:
    Map blue-red-purple US presidential election results by county.

mailtests:
    Messages with attachments for testing viewmailattachments.

motioncam:
    An automated motion-sensitive camera driver, for Raspberry Pi or
    other machines. Requires pycamera module.

pagewatch:
    Watch a website and summarize the important parts of it.
    For instance, watch a page to see when something you want to buy
    is in stock, or to see when tickets for an event become available.

pycamera:
    A wrapper that can drive a Pi camera module, a USB webcam or an
    external gphoto2-compatible camera using similar APIs.

rpi:
    Scripts for the Raspberry Pi. Mostly wrappers to talk to sensors.

weather:
    Some attempts at accessing various weather APIs and plotting
    statistics.

--------------------------
Scripts in this directory:
--------------------------

androidfiles.py:
    A Python module to make it easier to list files on an
    Android device using adb, and copy individual files to the device.
    Can also synchronize two directories, trying to notice when files
    need to be moved rather than removed and recopied.
    Used by podcopy.

angle_properties.py:
    Example code for how to use properties in Python,
    and also how to do operator overloading.
    Defines an Angle class, which isn't meant to be particularly
    useful, more a cheatsheet for learning properties.

animate.js:
    Simple code for animating a series of images in javascript.

aptsearch: Easier way of using aptitude search, which then calls
    apt-file search if aptitude doesn't find any matches.

aptver:
    Search for Debian packages, installed or uninstalled, matching
    a pattern, and print out versions and descriptions.

attention.py:
    Keep a log of which virtual desktop you're on and the current idle time.
    The idea is to run it all day in a loop, and plot how often you're
    context-switching.

balanceparens.py:
    A very simple parenthesis (and brace and bracket) counter.
    Why isn't there a better one already on Linux?

birdcodes.py:
    Give common name for those obscure 4-letter bird codes.

bdump:
    Ascii/hex/decimal dumps of any file. Like od but much more readable.

blogtouch.py:
    In a pyblosxom hierarchy, find files matching a pattern and
    set the time on them to be ten years in the past, so that the next
    pyblosxom.cgi --static --incremental will update them without
    needing to change the date of the source file (which would
    move it to the top of the blog).

bookfind.py:
    List books by author optionally year of publication,
    to find new books by particular authors.

breaktime.py:
    Reminds you to take a break from the computer every half-hour,
    looking at idle time to see if you're already away.

browser_cookies.py:
    Unpack a firefox or chrome/chromium cookie file to show your cookies.

cachefile.py:
    A general class for cases where you need to fetch data from some
    slow process, like a web API, and cache it locally for speedier access.
    Caches in CSV format.

cairodrawwin.py:
    A base for building Python3 GTK3/Cairo apps using a drawing area.
    You can import and subclass it, or use it as a template.

camset.py:
    A wrapper for v4l2-ctl, to allow camera capability queries that
    don't scroll for three pages, and settings based on profiles so
    you don't have to fumble around with history or aliases for five
    different commands when the camera resets in the middle of a
    session.

cellaut.py:
    Simple cellular automata simulation in Python.
    Slow, not something you'd want for real simulations.

censusdata.py
    Functions to help in parsing US decadal census data.

chatsounds.py:
    An xchat Python script to handle nick (and other special-case) sounds.
    Also a rare example of an xchat Python script.

check-monitors
    A shell script to probe for connected monitors and use xrandr to
    activate the right one. Obsoleted by monmon.py.

cleanhtml.py
    Clean up HTML edited by gmail users -- remove font/color/style tags
    and such. Then prettyprint (somewhat -- add newlines, at least).
    Minimal, but serves as an example for how to clean up HTML:
    every source may have different types of clutter, so you may need
    to customize for whatever source you're trying to clean up.

colormatch.php:
    Find the nearest matching color name. http://shallowsky.com/colormatch/

confusables.py:
    Detect similar-looking letters, e.g. 𝒂pple.com where the first
    character is actually unicode "MATHEMATICAL BOLD ITALIC SMALL A".

countsyl:
    Experiments in counting likely syllables in words, giving best guess
    and likely max. Not particularly reliable, but I haven't found a
    better way.

covid-timeseries.py:
    Plot COVID data from the Corona Data Scraper.
    Because I'm as obsessed by COVID-19 as everybody else.

crawler.py:
    Crawl a website (typically localhost) looking for bad links.

decodemail.py:
    Decode email lines encoded in RFC 2047 format, printing only lines
    that match a specific header (Subject:, From:, etc.).
    Useful for writing procmail rules that handle nonascii charsets.

delaytest.cgi:
    A CGI script in Python that spits out some text slowly, with delays.
    Use it on your webserver (or localhost) to test things like parallel
    downloads to make sure they work with slow servers.

domaincheck.py:
    Check for expiring domains, using python-whois.

diff2rss:
    Diff two files and output RSS of the differences.
    A possible (though poor) solution for web pages that don't offer RSS feeds.

diffiles.py:
    Find files that differ between two directories -- even if
    they've been moved to different places in the directory structure.

dirsneeded.py:
    Run a program inside strace and check what files and directories
    are needed during the run. Useful for setting up a chroot.

docx2html, doc2html have been replaced by word2html.

eggtimer.py:
    Pop up a reminder window in a specified number of minutes.
    Also has an example of how to do full-duplex interprocess
    communication with Unix domain sockets.

fakefileman.py:
    A way to have the XDG desktop system pop up a terminal when
    it thinks you want a file manager.
    Useful for things like Zoom, Firefox's download manager,
    some Wine programs, etc.

falls_between.py:
    given a day of the week ("tue") or a day of the month (15),
    did it occur between two given dates?

fincompare.py:
    Download financial data from Yahoo finance and plot lots of
    different funds on the same scale.
    Can also take data from a local spreadsheet or other source.

firefox-zoom.py:
    Show which sites have saved zoom settings in firefox,
    or delete all saved zoom settings.

fix_agenda.py:
    Are you in an organization where everyone lives and dies by MS
    Word, emailing dozens of Word documents around for every meeting,
    and you'd like to take that pile of documents for the next meeting
    and convert them to HTML so they can be shared on a website and
    viewed in a web browser? That's what this script is for.
    Requires word2html (in this git repo), python-mammoth, unoconv
    (a part of Libre Office), and BeautifulSoup.

fontasia:
    A Linux font chooser and categorizer.
    Also works as a GIMP plug-in font chooser.

fotogr.py has moved to my metapho repository.

gitbranchsync.py:
    Display git branches and what they're tracking,
    check whether a repository needs pushing upstream,
    and optionally make local branches track remotes of the same name.
    A solution to the problem of git making it idiotically difficult
    to keep several machines in sync with the branches in a git repository.

gmail-oauth-tokens.py:
    A script to manage GMail OAuth2 tokens,
    derived from getmail6's getmail-gmail-xoauth-tokens
    which itself is derived from Google's orphaned oauth2.py.

gpreso:
    A lightweight presentation program for displaying HTML slides.
    Python2/GTK/webkit version. (Previously called preso.)

gtktreeview.py:
    An example of how to change the selection programmatically in
    a scrolled GTK3 TreeView. Because this isn't documented AT ALL.

gquickbrowse:
    Minimal browser: an older version of quickbrowse written in
    python2-webkit-GTK, now orphaned and no longer available on newer
    releases. If you don't pass an argument but there's anything in
    the X selection when you run it, it will use that as initial argument.

guitarflash.py:
    Play and display guitar chords, intended for a "flash cards" sort
    of learning tool for beginning guitar students.
    Uses command-line sox play to play chords and metronome ticks.

headerfooter.py:
    Take standalone HTML files and add customized headers and footers
    to make them fit in with a website, replacing anything up to and
    including <body> and following and including </body>

htmlmail.py:
    Compose and send HTML mail, including image attachments.

icalview.py:
    Interpret an .ics calendar attachment, including timezones.

intel_backlight.py:
    Tiny script to adjust the backlight on an Intel-based laptop
    that uses /sys/class/backlight/intel_backlight.

imageviewer_gtk2.py:
    A generic image viewer widget and image viewer window that can
    be imported into other python programs.

imageviewer_gtk3.py:
    A generic image viewer widget and image viewer window using GTK3
    via gi, so they can be used with Python 3.
    Offers translucency (with a deprecated method) but not clickthrough.
    There is no GTK4 imageviewer; when GTK4 started insisting on client-
    side decorations I gave up on it. See imageviewer_tk.

imageviewer_tk.py:
    An image viewer written in Tkinter. I got tired of always having
    to rewrite everything as GTK's APIs change, and decided to migrate
    most of my apps, gradually, to TkInter.

imgcpy.py:
    Copy a file, suitably downsized, to the clipboard in PNG format
    so it can be pasted by an app like Discord that uses image pastes.

ipcsimple.py:
    A module intended to make it easy to do the sort of simple
    inter-process communication used by eggtimer and breaktime,
    like querying a running process to get information from it.

ipsearch:
    Search for live hosts on the current net, or hosts with a
    particular port open..
    Useful for things like finding that Raspberry Pi you just booted.
    Also has some useful network wrappers like ping, arp, fping,
    list network interfaces and find current network.

ipscan:
    A Bourne shell (sh) net scanner/pi finder uses nmap.

keyreader.py
    Read characters (keystrokes) one at a time (cbreak mode),
    with or without blocking, with or without echo,
    on Unix/Linux/Posix platforms.

kitfox-patch
    A simple shell script to update Firefox's browser/omni.ja file
    with custom patches when a new version of Firefox comes out.
    Stopped working some time in 2020, alas, and I haven't found
    any way to apply these patches to more modern Firefoxes.

langgrep:
    Search (grep) for patterns only within scripts written in a
    specified language (e.g. in all python scripts).

legistarparse.py:
    An experiment in using pdfminer to parse legistar PDFs,
    for use with losalamosmtgs.py.

losalamosmtgs.py:
    Scrape the Los Alamos legistar page to find out about upcoming
    county meetings and find out what's on their agendas.

louder:
    Increase alsa system volume by a small increment. Symlink it to
    another name like "quieter" and it will decrease instead.
    Bind those to volume increase/decrease keys on your laptop.
    and set it as the system wallpaper.

mac_lookup.py:
    Map MAC network addresses to manufacturers.
    Useful for things like scanning your network to find your Raspberry Pi.

mailgrep:
    Search for patterns in mailboxes, whether they're mbox (uses grepmail)
    or maildir (uses grep). Bring up mutt on a folder with matching messages.

masq:
    Set up IP masquerading to talk to another computer, such as a
    Raspberry Pi, over eth0 while using wlan0 for the internet.

mbsyncd:
    isync/mbsync is a good IMAP sync program, but for some bizarre reason
    it doesn't offer a daemon mode where it syncs regularly forever.
    So mbsyncd is a daemon wrapper for mbsync.

mnist.py:
    A simple example of a neural net using keras/tensorflow.

moncycle.py:
    Cycle between connected monitors (e.g. laptop display to HDMI).
    Can be bound to XF86Display in your window manager.

monmon.py
    Probe for connected monitors and their sizes and relative positions.
    Find "orphaned" windows (windows that are invisible offscreen)
    and move them back to a monitor that's still connected.
    Replaces and extends check-monitors.

motionphoto_split.py:
    Separate the jpeg and video from a Google MotionPhoto file
    from a Pixel phone.

mouseevent.py:
    Read mouse events from /dev/input even if not running X,
    using the python evdev module. Linux only.

mpl_smart_dates.py:
    A smart date locator (ticker) and formatter for matplotlib
    that can autoadjust according to the amount of time the axis
    spans. Matplotlib's AutoDateLocator does this for major ticks,
    but doesn't offer the option of minor ticks.

multiplot3d.py:
    3D plots in matplotlib of a sequence of functions are so poorly
    documented, it took me ages to get one working. Here's a simple example.

mpcdemo.py:
    A tiny demo of how to use multiprocess communication in the
    multiprocessing module to start a worker thread and pass messages
    to it.

mplayerfilenames.py:
    Run mplayer on a list of files, showing each filename as a subtitle.
    Helpful for triaging video files.
    Creates .srt subtitle files which will be left in /tmp
    in case you want to view the same videos repeatedly.

musicplayer.py:
    A little toy music player in Python, illustrating how to play audio.
    Actually more than a toy now -- I use it as my main music player.

namedpipe.py:
    Illustrates how to use named pipes. Just a demo, nothing useful.

noblank:
    Turn off screen blanking. Run this when you're giving a presentation.
    Linux-specific.

nokeywords:
    Find directories beneath the present one that don't contain a
    Keywords or Tags file -- i.e. image directories that need to be tagged
    with metapho.

nonblock.py, nonblock-py3.py:
    Examples of how to do nonblocking input in Py 2 and Py 3.
    For code that works in either version, see termsize.

noteplayer.py:
    Spun off of play_chord.py: a more general pygame-based  note generator
    usable asynchronously, like from a Raspberry Pi piano keyboard.

openlink.py:
    Search for the biggest firefox window and use Linux utilities like
    xdotool, wmctrl and xwininfo to cause the firefox window to browse
    to a given URL in a new tab.

pathmatch:
    Search your executable path to find programs matching a string.
    Like str<tab> except you can match patterns anywhere in the
    string, not just at the beginning.

phonewords.py:
    Map phone numbers to words, or vice versa. Find phone numbers
    that map to more than one word.

pisoothe:
    Play sound samples (e.g. soothing white or pink noise samples)
    in a loop, reacting to key events for volume control or to skip
    to the next sample.

play_chord.py:
    Example of how to generate and play chords using Python and NumPy.
    Can also play strings of notes, or can play notes from the keyboard.

podcopy:
    Copy podcasts (fetched with podget) to a mounted mp3 player or directory,
    or to an Android device via adb.
    Remove characters in filenames that would cause problems on vfat
    filesystems, and ensure filenames are unique and files aren't dups.

prettysoup:
    Prettyprint a BeautifulSoup object. Customizable, and avoids
    problems like BS4 prettify() adding spaces in the middle of words.

pulsehelper.py:
    A script to list or switch audio inputs/outputs
    on a system that uses PulseAudio.

pyclip:
    Read the X clipboard.

pykey.py:
    Simulate keypresses under X. A simpler, Python version of
    Crikey, http://shallowsky.com/software/crikey/

pyplay.py:
    A class to play sounds asynchronously by calling aplay (or an app
    of your choice). Don't repeat a sound that's already playing.
    Used in the xchat script chatsounds.py.

pyruler:
    An X pixel ruler. Useful for measuring things like icon sizes.

pysync:
    Call rsync, but check first what would be changed, and save the
    password between the two calls.

python-cheatsheet.py:
    My personal cheatsheet and syntax reminders for useful Python features.

qdroptarget.py:
    A little PyQt window that acts as a drop target and can run a
    program on text that's dropped or middlemouse-pasted there.

qhtmlprint:
    Convert a set of HTML slides (e.g. in navigate.js) into PDF.
    For turning an HTML presentation into a PDF slide deck.
    Unfortunately the QtWebkit libraries this depends on
    seem to be abandoned; see qslide2pdf for a newer version.

qslide2pdf:
    Convert a set of HTML slides (e.g. in navigate.js) into PDF.
    For turning an HTML presentation into a PDF slide deck.
    Uses Qt5's QWebEngine and PyPDF2. We'll see how long that's supported.

qpdf.py:
    A simple PDF viewer for Python-Qt5 using popplerqt5.

qpreso.py:
    A lightweight presentation program for displaying HTML slides.
    A Python3/Qt5/QtWebEngine version, since python-gtk-webkit
    (used for the older gpreso) is no longer supported.

quickbrowse.py:
    A minimal browser in python-qtWebEngine. Fast and light,
    supposedly anonymous, no storage of cookies or other identifying
    information. Useful for things like viewing URLs from mutt or
    checking websites that don't work well with a Firefox/NoScript profile.
    Can be used standalone (ln -s /path/to/quickbrowse.py ~/bin/quickbrowse)
    and/or as a python module for derived browser-based programs.

quotekiosk.py:
    Display a series of plaintext quotes in a large, maybe fullscreen,
    window, as large as possible to fit in the window, changing the quote
    every so often.

qquotekiosk.py:
    Display a series of quotes or other content from HTML files,
    as large as possible to fit in the window, changing the quote
    every so often, using qpreso.py for the display window.
    Uses qpreso.py for the HTML display window.

randombg:
    Choose a random background (wallpaper) from a directory named based on
    current X resolution. So you can have different sets of backgrounds
    sized for laptop screen, external monitor etc.

randomgeom.bash:
    Obsoleted by randomgeom.py.
    bash (or zsh or other modern shell) script to generate an
    X11 geometry argument with a random position along the edge
    of the monitor, like 165x65-1899-0.
    I used this to position dclock in different places each day to
    avoid monitor burn-in; to anyone else, it's mostly useful as an
    example of how to do random number calculations in a shell script.

randomgeom.py:
    A smarter, more general version of randomgeom.bash:
    generate an X11 geometry argument with a random position along the
    edge of the monitor.

randomline:
    Print a random line from a file. Useful for things like choosing
    a random wallpaper or random system beep tone.

readkey.py:
    Read a key press from the keyboard without waiting for a newline.
    For CLI programs: doesn't need X, tkintr, pygame or similar libs.

remindcgi.py:
    CGI: Run remind from a "remind.txt" source file in the same directory,
    showing either upcoming reminders or all events for the next
    week/month/ever.

remindexpire.py:
    Expire events in remind.txt that are now in the past,
    saving them to "archived.txt" in the same directory.

repo-pullpush:
    Modify a git repository, taking the existing URL, making that
    the pushurl and adding a passwordless pull URL alongside it.

rise_set_az.py:
    Find the times when the sun or moon will rise or set at a specific
    azimuth, within a specified phase range, from a specified location.

scrubjay:
    Demo of how to use a few different Twitter API.
    No UI.
    Two variants: scrubjay-tweepy (tweepy) and scrubjay-p-t (python-twitter).

searchhistory.py:
    Given places.sqlite3 from a Firefox profile,
    display all the search terms that have been passed to Google.

serio:
    A proof-of-concept serial terminal app in Python.
    Don't take this seriously; it's not meant as a real app (yet?)

shorturl:
    Use a URL shortening service to get a shorter URL.
    Can take input from the current X selection, and automatically
    select its output for pasting into a browser or Twitter window.

shorturl-gi3:
    shorturl ported to python3, GTK3 and GObject introspection (GI).
    Most useful as an example of how to use GI-GTK3.

sieve.py:
    Visualization of the Sieve of Eratosthenes, using Python curses.
    (Not very useful, but makes a good curses example.)

sinkto:
    Synchronize files in a list between two machines.
    Good for syncing updated scripts, dot files, documents,
    email folders etc. to a laptop when preparing for or returning
    from a trip, or just for keeping several machines in sync.

spiderchart.py:
    A tiny example of spiderweb/radar/radial charts in plotly.

speedtestserver:
    A mini http server that can introduce delays or other glitches
    that are useful for testing http download clients.
    Unfortunately, it's only single-threaded which makes it
    not very useful except as an illustration of how to make
    a mini server in Python. Use delaytest.cgi for actual testing.

strongpasswd.py:
    Generate a strong random password.
    Firefox's is useless because it doesn't include punctuation
    (see bug 1559986), while password managers tend to assume a wider
    set of punctuation than most real websites allow.

tee.py:
    A file-like class that can optionally send output to a log file.

termcolors:
    Display ANSI colors for the current terminal.

termsize:
    Set stty size to the size of the current terminal window,
    e.g. when logged in to an embedded Posix device over a serial line.

text_wrapped_image.py:
    Demo of how to add wrapped text to an image using PIL/Pillow.

tklabelimage.py:
    Mini example of how to display text and images together in a label
    in TkInter, and of how to conditionally import Tk in python2 or 3.
    Doesn't do anything useful.

towebhost.zsh:
    Handy script to figure out which web host files need to be
    uploaded to, based on their local directories.

transimageviewer_gtk2.py:
    OBSOLETE because GTK2 is.
    A translucent image viewer (inheriting from imageviewer.py)
    that passes clicks and other events through to the window below.
    Useful for things like saving GPS waypoints from a map image.
    Also offers a rare example of how to use the X11 SHAPE extension
    from pygtk2, including input shapes.
    Alas, I haven't found a way to do this in GTK3.

transimageviewer_tk.py:
    An attempt at a translucent clickthrough image viewer in tk,
    which DOES NOT allow clickthrough (yet?)
    It tries to use an xlib technique for that, but it doesn't work
    and I'm not sure why not (the same code works in transimageviewer_x.py).

transimageviewer_qt5.py:
    A translucent image viewer with clickthrough, implemented in qt5.
    Can't be moved once it's mapped, but otherwise works.

transimageviewer_x.py:
    A translucent image viewer with clickthrough, implemented in Xlib.
    This one is the best of the bunch because you can move it around
    if you grab it near the top center of the window.

turtlefrac.py:
    Draw fractals (specifically a Sierpinski triangle)
    as an example of how to use Python turtle graphics.

tweet:
    Play birdsong from a specific bird. Requires ripped tracks from a
    bird song CD: for example, the Stokes, Cornell or Peterson collections.

unidecode.py:
    Print the codepoints and names for unicode characters.

urldownloader.py:
    Maintain a queue of URLs to download asynchronously to local files.

us-wars.py:
    Plot a Gantt chart of years of wars the US has participated in.
    Also serves as an example of a matplotlib Gantt chart.

vcalparse:
    Parse and print an icalendar/vcalendar/ICS file using python-vobject.

pip_upgrade.py:
    Upgrade all Python packages in the current virtualenv.
    Don't try to upgrade anything in the system Python directories.

vidtriage.py:
    View a list of videos in sequence, tagging them with numbers
    or 'd' (for delete) to triage which ones belong to which category.

viewhtmlmail.py: OBSOLETE: see viewmailattachments.py

viewmailattachments.py:
     (Replaces the older scripts viewhtmlmail.py and viewmailattachments)
     Show HTML messages in an external browser, including embedded images.
     Also show attachments in MS Office formats or PDF after
     converting them to HTML.

vol:
    A simple python-gtk volume slider. Much simpler and smaller than
    alsamixergui.

vote411export.py:
    A script to take an exported format from the LWV VOTE411 site,
    parse it, and produce clean outputs for a printed Voter Guide.

weborphans:
    Check a web site (perhaps localhost) against a local mirror,
    checking for broken links and orphaned files.

whenupdated.py:
    Keep track of the content of a URL, and when it changes, store
    the new version with a timestamp so you can see when a page is updated.

wikifollow.py:
    Starting with any Wikipedia page, follow the first real link
    on each successive page until it arrives at a target endpoing.
    The theory is that everything always ends in "Philosophy".

wikitable.py:
    Parse a data table from a Wikipedia page, outputting CSV.

word2html.py:
    Convert .docx or .doc to prettyprinted HTML.
    Uses python-mammoth for docx, and either uniconv + python-mammoth
    or wvHtml for .doc. Uses BeautifulSoup for prettyprinting.

worddiff.py:
    Convert two files from MS Word (.doc or .docx) to text or markdown,
    then run a diff on those two files, to see if the content of the
    Word file has changed.

wordview:
    Convert a doc or docx file to html, and call a new firefox window on it.
    Predecessor to (and simpler than) viewmailattachments.

wpnet.py:
    A wrapper to make it easier to use wpa_cli to manage known wireless
    networks.

xbright:
    Adjust brightness of an X11 display up or down using xrandr.

xlsrd:
    Display an .xls spreadsheet as a formatted ASCII table, using
    the xlrd package to parse the spreadsheet.

zoomrecordings.py:
    List recordings available to download, with URLs and sizes.
    Gives a way to download zoom recordings without needing a browser
    (if you have the JWT token for the Zoom account in question).
    Super-simple example of the Zoom API.


--------
Testing:
--------

Only a few of the scripts have tests. Run all tests with:

python3 -m unittest discover