TTK's Code Closet


Updated 2013-05-23

 
I have been asked by three people now to get a github account for these projects.  That will be happening in the near future, and a link made from this page to the github project(s).  dy and select will also be updated soon.
 
The quality of this code tends to be very irregular.  Sometimes a personal tool becomes a professional tool (qv, mssh, extants) and I clean it up, but just as often I will rewrite it from scratch instead to be cleaner, more generalized, and more powerful (qv, calc3 and the extants builder).
 
All of this code is freely available for anyone to download, distribute, and use.  Some of it lacks any kind of license, but I have licensed some of it under a very weakened BSD-style copyright.  This copyright puts the user of the code under virtually no restrictions -- you are free to use, distribute, and modify the code, even in a commercial product, with no obligation to give me or anyone else credit or compensation, no obligation to contribute changes back into the public, and no obligation to provide source.  It is a nonviral license.  Its sole purpose is to prevent some third party from infringing on the rights of others to download, distribute, use, or modify the public copies of the software.  I like to see my stuff used, and if commercial software developers can use it to make a better product, it's cool by me.  I'll probably benefit from it (indirectly) some way or another; all technologies are connected to some degree.  So there are no restrictions which might discourage a commercial software developer from using this code.
 
    Click -- Go Back to TTK Ciar's Home Page
 
    Update 2013-05-23:  mssh updated
    Update 2013-05-21:  isabella updated
    Update 2008-11-20:  calc3 updated
    Update 2008-08-19:  select updated
 
 
 
mssh
Multi-target SSH remote command execution tool (perl)
Last updated: 2013-05-23
 
"mssh" will make ssh connections to several remote hosts in parallel, run a command on the remote hosts (and/or copy files to those hosts), and stream their outputs back to the user.  It is scalable (worked fine on archive.org's 2000 hosts), handles large command outputs gracefully (even infinite outputs), and provides many options for:
  • selecting logical subsets of target hosts,
  • saving command output to host-specific files,
  • prepending command output with user, node, and time,
  • throttling or retrying ssh connection attempts,
    ... and more.  See the README for a full list of options.
     
    Fan-out (making connections to a subset of the target hosts, then making the remaining connections from those hosts instead of the host from which mssh was run) is not yet implemented, but it can be "faked" with adroit use of --up, and making the command itself an mssh command.  This technique allowed mssh to scale to clusters of 20,000 hosts at a previous employer.  Integrated fan-out is next on my list of features to develop for this tool.
  • isabella
    Lucene-like pure-python search index (python)
    Last updated: 2013-05-21
     
    THIS CODE IS UNDER DEVELOPMENT AND NOT YET READY FOR USE!
     
    "isabella" is the first in a series of projects to re-implement certain useful infrastructure components in python.  Slated for replacement: Lucene, Cassandra, and Zookeeper.  The objective is to provide stable, reliable, and scalable replacements, but not exact API-level compatability.
    Calc v3
    Calculator and command shell (perl)
    Last updated: 2008-11-20
     
    THIS CODE IS UNDER DEVELOPMENT AND NOT YET READY FOR USE!
     
    "Calc3" is a re-implementation of ye olde perl calculator (qv calc in this codecloset).  Calc1 and calc2 were developed very haphazardly and without any thought to maintainability or expandability, so calc3 is an attempt at a cleaner implementation with an eye towards future extension, and inclusion and integration of similar interactive clients I've cobbled together in recent years.  It introduces the concept of "modes", so that depending on which mode the calculator is in, it could interpret C, perl, SQL, or shell commands, though at the present only the "perl" mode is even marginally functional.  Though it is still primarily a calculator, in time I aim to make calc3 my default shell.
     
    Please go read the documentation (such as it is) for calc2 first. Calc3 behaves like calc2 in its default mode.
     
    I have also talked about calc3 from time to time on my journal.
    select
    Structured data transformation utility (perl)
    Last updated: 2008-08-19
     
    "select" is a commandline tool similar in concept to SQL's "select" function. It reads structured data on stdin in a variety of formats (csv, tab-delimited, json, xml, and others), and writes to stdout a subset of that data, according to the rules specified by the user. It is capable of transforming between most supported formats, so that one might use it to convert from json to xml, or from csv to html, for instance. It is still very much a work in progress, and transformations between some format types is sketchy (if not outright broken).
    dy ver 1.3-2006-01-26
    file heirarchy utility (perl)
    Last updated: 2006-01-27
     
    "dy" is like the bastard child of "ls" and "du" (think du with a little extra). It will traverse the given files (or all files in PWD if no files are specified), recursively descending through directories if any, and emit a one-line summary of information about each file. Output is intended to be easily machine-readable first, and human-readable a distant second.
    orcus ver 0.1-2001-11-09
    chat utility (C)
    Last updated: 2001-11-09
     
    In my spare time I like to develop a client application for the ICB chat system.  Releases are erratic.
    Depends on stdtypes
    proto2html ver 1.0
    Web publishing utility (Perl)
    Last updated: 2001-08-11
     
    Converts (most) plaintext documents into html with appropriate formatting.  Allows notational shortcuts for embedding href links, et al.
    codefrag ver 1.0
    Multi-purpose programmable utility (Perl)
    Last updated: 2000-08-09
     
    Dirty (but useful!) hack that lets you type C code into a command line and have it executed immediately, akin to "perl -e '..'" functionality.  Also lets you mix (sort of) C and perl on the command line.
    nlfsr ver 1.1
    Nonlinear Feedback Shift Register function library (C)
    Last updated: 2001-01-19
     
    Implementation of an nlfsr adt -- a good pseudorandom number generator.
    extant ver 1.2
    "Extant" (blocked list) function library (C)
    Last updated: 2002-07-21
     
    General-purpose dynamically-resizing static arrays of integers, characters, pointers, or structs.  Performs automatic bounds checking, et al.  I use these all the time in personal and professional work.  Especially useful when you need a buffer, but you don't know how big, and you want to avoid overflow.  Provides functionality similar to linked list, but allows random access and keeps memory overhead and access latency low by keeping all data compacted into static blocks.  Obsoleted this with extantbuilder (qv) utility.  Original extants is now folded into misclib (qv).
    fsbench ver 1.0
    Filesystem benchmarking utility + result lists (Perl)
    Last updated: 2000-07-02
     
    Benchmark script some people have used to profile the performances of various filesystems on numerous platforms.  More recently I wrote an improved version and ran it on various platforms as well.
    misclib ver 0.2_beta
    Function library (C)
    Last updated: 2010-05-04
     
    Modest collection of various useful C functions.  I've got more where that came from, but I need to take the time to collect them and stick them in.  Some are focussed on increasing reliability, others are focussed on rapid application development, and others are high-performance tools.  Many of the C functions formerly listed on this web page have been incorporated into misclib.
    Depends on stdtypes
    extantbuilder ver 1.2
    Utility for generating C function libraries (Perl)
    Last updated: 2002-07-18
     
    Utility for automatically synthesizing C function libraries implementing type-specific extants (qv).  Allows customization of storage type, degree of complexity (ie, can be as simple as the extants in the generic extants library, for low memory overhead.  Future versions will allow extants as sophisticated as desired).  Work in progress -- future versions will support multithreaded access, autoserialization of data, etc. (Name changed on account of too many people showing up here looking for a Mklinux filesystem tool of the same name.)
    qd ver 4.0
    Binary viewer - function library and stand-alone utility (C)
    Last updated: 2001-01-28
     
    QuickDump is like a customizable "dd" on steroids.  It displays binary data in an easy-to-read format.  Also included is qdlib, a function library that makes it easy to embed qd-like functionality into software (excellent for debugging purposes).
    slice ver 1.0
    File-copying/organizing utility (Perl)
    Last updated: 2000-05-11
     
    Utility for easily chopping up filesystem heirarchies into chunks no larger than N bytes and moving them into subdirectories.  Written for a friend who needed something to help him back up *huge* filesets onto burnable cdroms.
    calc ver 2.5
    Calculator (Perl)
    Last updated: 2005-10-25
     
    Handy-dandy perl calculator.  Includes a breakout function library for extensibility, a stack, command history, virtual ticker tape, saving/restoring of states to disk, and interpretation of stack references.  Also useful for testing perl expressions (since the user's input is simply munged a little and then passed to eval()).  Also includes many preloaded functions and some constants.
    stdtypes ver 1.0.1
    Utility and header file (C)
    Last updated: 2001-08-24
     
    Utility which inspects the local host (a la autoconf) and generates a "stdtypes.h" file appropriate for the platform.  stdtypes.h defines and typedefs various useful known-length types and other data which can be used to trivially make software more portable between platforms, and help facilitate communication through files, pipes, or sockets between platforms with different data type sizes and/or byte orders.  I use this one all the time, too.  IMHO it's a lot better than the one the ANSI committee came up with.

     
     
        Click -- Go Back to TTK Ciar's Home Page