██████   ██████  ████████ ███████
██   ██ ██    ██    ██    ██
██   ██ ██    ██    ██    ███████
██   ██ ██    ██    ██         ██
██████   ██████     ██    ███████

◎ Config files and utility scripts

Overview

This repository contains configuration files and bash scripts to automate various development tasks. The files are organized into directories based on categories like Python, Git, Zsh, and more.

Project Structure

The repository is structured as follows:

└── dots/
    ├── bash
    │   ├── builds
    │   │   ├── docker_hub.sh
    │   │   └── pypi.sh
    │   ├── common
    │   │   ├── clean.sh
    │   │   ├── run.sh
    │   │   └── test.sh
    │   ├── file-ops
    │   │   ├── aggregate_docs.sh
    │   │   ├── chunk_docs.sh
    │   │   ├── modify_filenames.sh
    │   │   └── move_directory.sh
    │   ├── install
    │   │   ├── local_dependencies.sh
    │   │   ├── local_dependencies_uninstall.sh
    │   │   ├── micromamba.sh
    │   │   └── pyflink.sh
    │   └── project-setup
    │       ├── create_directory_structure.sh
    │       ├── create_python_project.sh
    │       └── tree_to_directory.sh
    ├── git
    │   ├── .gitcommit_template
    │   ├── .gitconfig
    │   ├── .gitignore
    │   └── .gitignore_global
    ├── make
    │   ├── Makefile-poetry
    │   └── Makefile-uv
    ├── python
    │   ├── .ruff.toml
    │   ├── pyproject.toml
    │   └── pytest.ini
    ├── vscode
    │   └── settings.json
    └── zsh
        ├── .zprofile
        └── .zshrc

Project Index

Select a submodule to explore its contents:

DOTS/

python

pytest.ini	Configure pytest settings for test coverage and reporting, ensuring tests meet a minimum coverage threshold.
pyproject.toml	- Generates README files using large language model APIs, enhancing developer productivity and documentation quality - Integrates with various tools for seamless workflow automation and markdown generation - Facilitates efficient project communication and collaboration through standardized README templates.
.ruff.toml	Define codebase linting and formatting rules using the provided configuration file for Python projects.

make

Makefile-poetry	- Manage Poetry environment, install dependencies, and generate requirements files for main and development dependencies - Display help information for Makefile targets.
Makefile-uv	- Facilitates installation and management of project dependencies using the uv tool - Includes commands for installing dependencies in editable mode, locking dependencies, syncing environment, creating a virtual environment, and displaying help information - Enhances project development workflow by streamlining dependency handling tasks.

zsh

.zshrc	- Sets up the interactive shell environment by configuring themes, plugins, aliases, functions, paths, and integrations - Loads scripts, sets history settings, and enables FZF keybindings - Includes custom functions and autocomplete settings for enhanced shell usage within the project architecture.
.zprofile	- Sets up the development environment for login shells by configuring paths and environment variables - Includes settings for Homebrew, Python development, local binaries, XDG Base Directory Specification, and development directories - This file plays a crucial role in ensuring a consistent and efficient development setup across the codebase architecture.

vscode

settings.json

- Configures various settings for the Visual Studio Code editor, including themes, font sizes, language-specific formatters, and extensions - Manages preferences for workbench, editor, terminal, Git, file associations, and language-specific settings - Enhances productivity by customizing the editor environment for efficient coding and collaboration.

git

.gitcommit_template	Define commit message template for clear and concise communication of changes made in the project.
.gitignore_global	Defines global Git ignore rules for common editor and system files to maintain a clean repository structure.
.gitconfig	Configure Git settings for streamlined collaboration and efficient version control.

bash

install

micromamba.sh	- Facilitates seamless installation and setup of Micromamba, a lightweight package manager, by determining the OS, downloading the latest version, making it executable, moving it to a global path, initializing it, and configuring default channels - This script streamlines the process for users to easily utilize Micromamba across different operating systems.
local_dependencies_uninstall.sh	- Perform local environment cleanup by uninstalling various dependencies like pyenv, goenv, Node.js, Poetry, Git, Git LFS, kubectl, kubectx, and Helm - Additionally, clean up configuration files to ensure a fresh start - The script logs each step and the total cleanup time, providing a streamlined process for resetting the local environment.
pyflink.sh	- Automates PyFlink setup by checking and installing Java 11, Python 3.7, and downloading PyFlink - Sets environment variables and aliases for zsh, enabling easy PyFlink usage - Streamlines the initial setup process for PyFlink integration within the project architecture.
local_dependencies.sh	- Automates local environment setup by installing essential tools like Oh My Zsh, Homebrew, pyenv, Go, kubectl, Helm, Node.js, Poetry, Git, and Git LFS - Updates shell configurations for seamless tool integration, ensuring a smooth development experience - Logs installation progress and completion time for user reference.

file-ops

chunk_docs.sh	Automates the process of chunking a document into 10 parts for easier management and navigation within the project's documentation structure.
aggregate_docs.sh	- Aggregate markdown files from a GitHub repository into a single document based on specified paths - Clone the repository, search for markdown files, concatenate their content, and output a consolidated markdown file - Handles various input parameters to customize the aggregation process.
modify_filenames.sh	- The code file modifies filenames in a specified directory by converting them to lowercase and replacing underscores with hyphens - This script helps maintain consistency in file naming conventions within the project structure.
move_directory.sh	Moves a specified folder to a destination directory if both exist; otherwise, displays appropriate error messages.

builds

docker_hub.sh	- Automates building, pushing, and publishing Docker images using Buildx - Sets up Docker Buildx, builds and pushes a single-platform image, then builds and pushes a multi-platform image - Displays the completion message with the published image name.
pypi.sh	- Automates PyPI package deployment by cleaning, building, and uploading distribution files - Integrates with PyPI API for seamless deployment of 'my-package'.

project-setup

create_python_project.sh	- Creates essential project directories, files, and configuration settings for a Python project - Sets up project structure, including source code, tests, and configuration files - Establishes a foundation for building and running the project with necessary components like logger, configuration classes, and main script.
tree_to_directory.sh	- Generates project directory structure and essential files for a vector SVG generator tool - Organizes core, templates, API, CLI, utils, tests, assets, docs, configs, examples, and root files - Ensures a well-structured foundation for the project setup.
create_directory_structure.sh	- Generates a directory structure for a markdown guide in the specified base directory - The script creates directories and files based on the predefined structure, ensuring proper organization for markdown content - This functionality aids in setting up a consistent and easily navigable layout for the markdown guide within the project architecture.

common

run.sh	- Automate environment setup and package upgrades in the project by running the provided bash script - The script activates the specified conda environment, upgrades pip, and logs the execution start and completion timestamps - This ensures a consistent and up-to-date development environment for the codebase.
clean.sh	- The clean.sh script provides commands to remove various artifacts like build, test, and Python files from the project directory - It helps maintain a clean and organized codebase by removing unnecessary files and directories.
test.sh	- Executes test coverage analysis for the project by running pytest with coverage reporting - It activates the 'readmeai' conda environment, sets source and omit directories, and generates coverage reports - The script ensures test coverage meets a minimum threshold for quality assurance.

Installation

To get started with the utility scripts, build the project from source:

1. Clone the repository:

❯ git clone https://github.com/eli64s/dots

2. Navigate to the project directory:

❯ cd dots

Running the Scripts

One of my favorite scripts to use right now is aggregate_docs.sh, which concatenates all files from a GitHub repository into a single file. I've found this script useful when working with language model APIs for providing the model with robust context and up-to-date information.

First, let's view the help message for the script:

❯ bash bash/file-ops/aggregate_docs.sh -h

Usage: bash/file-ops/aggregate_docs.sh [-r REPO_URL] [-p POSSIBLE_PATHS] [-o OUTPUT_FILE] [-n REPO_NAME] [-s TO_SEARCH]
  -r REPO_URL       URL of the GitHub repository to clone (required)
  -p POSSIBLE_PATHS Comma-separated list of possible paths to search for markdown files (required)
  -o OUTPUT_FILE    Name of the output markdown file (required)
  -n REPO_NAME      Name of the repository (required)
  -s TO_SEARCH      Pattern to search for markdown files (e.g., "*.md") (required)

To run the script, provide the following arguments:

❯ bash bash/file-ops/aggregate_docs.sh \
    -r https://github.com/pydantic/pydantic \
    -n pydantic \
    -o pydantic-docs.md \
    -p docs \
    -s "*.md"

Alternatively, make the script executable:

❯ chmod +x bash/file-ops/aggregate_docs.sh

And run it directly:

❯ ./aggregate_docs.sh \
    -r https://github.com/pydantic/pydantic \
    -p docs \
    -o pydantic-docs.md \
    -n pydantic \
    -s "*.md"

Advanced Configuration

Say you want to clone this repository and use the .zshrc file as your shell configuration, and have quick access to the function under the bash directory. We can set this up by creating symlinks from the repository to your home directory.

1. Create symlinks from your repository to your home directory:

   # Current repository location
   ❯ DOTS_DIR="/Users/<username>/GitHub/dots"
   
   # Create symlinks
   ❯ ln -sf "$DOTS_DIR/zsh/.zshrc" "$HOME/.zshrc"
   ❯ ln -sf "$DOTS_DIR/bash" "$HOME/.zsh/functions"

2. Add the following line to your .zshrc file to source the utility scripts:

   # -- GitHub/dots Repository Integration ---------------------------------------------
   DOTS_DIR="$HOME/Documents/GitHub/dots"
   DOTS_SCRIPTS="$DOTS_DIR/bash"
   
   # Function to load scripts from directory
   function load_scripts() {
   	local dir="$1"
   	if [ -d "$dir" ]; then
   		# Create functions for all scripts instead of aliases
   		for script in "$dir"/*.sh; do
   			if [ -f "$script" ]; then
   				local func_name=$(basename "$script" .sh)
   				eval "function $func_name() { $script \"\$@\" }"
   			fi
   		done
   	fi
   }
   
   # Load all script directories from dots/bash
   for category in $DOTS_SCRIPTS/*; do
   	if [ -d "$category" ]; then
   		load_scripts "$category"
   	fi
   done
   
   # Add custom functions directory to fpath
   fpath+=("$DOTS_DIR/zsh/functions")

3. Now you can run the scripts directly from your terminal:

   ❯ aggregate_docs -r https://github.com/pydantic/pydantic-ai \
   		 -p docs \
   		 -o pydantic-ai-docs.md \
   		 -n pydantic-ai \
   		 -s "*.md"

This setup maintains the repository structure while making all bash scripts accessible as commands in your shell.

Tip

The bash scripts directory contains a variety of scripts for automating tasks like environment setup, file operations, and codebase management.