Add new blog

src/content/posts/managing-tech-assets-common-lib-or-not/index.md

@@ -0,0 +1,115 @@
+---
+title: Managing Tech Assets - Is a Common Library a Good Idea? No
+published: 2024-07-17
+description: 'Managing Tech Assets - Is a Common Library a Good Idea? No'
+image: './cover.PNG'
+tags: ["Management", "Technologies"]
+category: 'Management'
+draft: false 
+---
+
+A Story - A Person Created a Common Library and Then...
+-------------------------------------------------------
+
+"Embarrassingly I introduced a "common" library, named as such, in a team environment a couple of decades back.
+I didn't really understand the dynamics back then of what could happen in a loosely-coordinated team setting in just a
+matter of months.
+
+When I introduced it I thought I made it clear and also documented that it's for things we'd all agree we find useful on
+a daily basis, that it's intended to be a minimalist library, and that the library should depend on nothing else besides
+the standard library so that it's as easy to deploy as possible in new projects. My thinking at the time was that it was
+our own little extension to the standard library for things that, in our particular domain, we found useful on a daily
+basis.
+
+And it started off well enough. We started off with a math library (`common/math*`) of routines which we all used on a
+daily basis, since we were working on computer graphics which was often heavy on the linear algebra. And since we were
+often interoping with C code, we agreed on some useful utility functions like find_index which, unlike std::find in C++,
+would return an index to an element found in a sequence instead of an iterator which mimicked how our C functions worked
+-- things of this sort -- a little bit eclectic but minimalist and widely used enough to remain familiar and practical
+to everyone, and instant familiarity is an extremely important criteria as I see it in trying to make anything that is
+"common" or "standard" since if it truly is "common", it should have that familiar quality about it as a result of its
+wide adoption and daily usage.
+
+But over time the design intentions of the library slipped out of my fingers as people started to add things they used
+personally that they merely thought might be of use to someone else, only to find no one else using it. And later
+someone started adding functions that depended on OpenGL for common GL-related routines. Further on we adopted Qt and
+people started adding code that depended on Qt, so already the common library was dependent on two external libraries.
+At some point someone added common shader routines which was dependent on our application-specific shader library, and
+at that point you couldn't even deploy it in a new project without bringing in Qt, OGL, and our application-specific
+shader library and writing a non-trivial build script for your project. So it turned into this eclectic, interdependent
+mess. Later on people even added GUI-dependent code to it.
+
+But I've also found by debating what should and shouldn't go into this library that what is considered "common" can
+easily turn into a very subjective idea if you don't set a very hard line rule that what's "common" is what everyone
+tends to find useful on a daily basis. Any loosening of the standards and it quickly degrades from things everyone finds
+useful on a daily basis to something a single developer finds useful that might have the possibility of being beneficial
+to someone else, and at that point the library degrades into an eclectic mess really fast.
+
+But furthermore when you reach that point, some developers can start adding things for the simple reason that they don't
+like the programming language. They might not like the syntax of a for loop or a function call, at which point the
+library is starting to get filled with things that's just fighting the fundamental syntax of the language, replacing a
+couple of lines of straightforward code which isn't really duplicating any logic down to a single terse line of exotic
+code only familiar to the developer who introduced such a shorthand. Then such a developer might start adding more
+functionality to the common library implemented using such shorthands, at which point significant sections of the common
+library become interwoven with these exotic shorthands which might seem beautiful and intuitive to the developer who
+introduced it but ugly and foreign and hard to understand for everyone else. And at that point I think you know that any
+hope of making something truly "common" is lost, since "common" and "unfamiliar" are polar opposite ideas.
+
+So there's all kinds of cans of worms there, at least in a loosely-coordinated team environment, with a library with
+ambitions as broad and as generalized as just "commonly-used stuff". And while the underlying problem might have been
+the loose coordination above all else, at least multiple libraries intended to serve a more singular purpose, like a
+library intended to provide math routines and nothing else, probably wouldn't degrade as significantly in terms of its
+design purity and dependencies as a "common" library. So in retrospect I think it would be much better to err on the
+side of libraries which have much more clear design intentions. I've also found over the years that narrow in purpose
+and narrow in applicability are radically different ideas. Often the most widely applicable things are the narrowest and
+most singular in purpose, since you can then say, "aha, this is exactly what I need", as opposed to wading through an
+eclectic library of disparate functionality trying to see if it has something you need.
+
+Also I'm admittedly at least a little bit impractical and care maybe a bit too much about aesthetics, but the way I tend
+to perceive my idea of a library's quality (and maybe even "beauty") is judged more by its weakest link than its
+strongest, in a similar way that if you presented me the most appetitizing food in the world but, on the same plate, put
+something rotting on there that smells really bad, I tend to want to reject the entire plate. And if you're like me in
+that regard and make something that invites all sorts of additions as something called "common", you might find yourself
+looking at that analogical plate with something rotting on the side. So likewise I think it's good if a library is
+organized and named and documented in a way such that it doesn't invite more and more and more additions over time. And
+that can even apply to your personal creations, since I've certainly created some rotten stuff here and there, and it
+"taints" a lot less if it's not being added to the biggest plate. Separating things out into small, very singular
+libraries has a tendency to better decouple code as well, if only by the sheer virtue that it becomes far less
+convenient to start coupling everything.
+
+:::tip
+Code deduplication has been hammered into me over the years but I feel like I should try it this time around.
+:::
+
+What I might suggest in your case is to start to take it easy on code deduplication. I'm not saying to copy and paste
+big snippets of poorly-tested, error-prone code around or anything of this sort, or duplicating huge amounts of
+non-trivial code that has a decent probability of requiring changes in the future.
+
+But especially if you are of the mindset to create a "common" library, for which I assume your desire is to create
+something widely-applicable, highly reusable, and perhaps ideally something you find just as useful today as you do a
+decade from now, then sometimes you might even need or want some duplication to achieve this elusive quality. Because
+the duplication might actually serve as a decoupling mechanism. It's like if you want to separate a video player from an
+MP3 player, then you at least have to duplicate some things like batteries and hard drives. They can't share these
+things or else they're indivisibly coupled and cannot be used independently of each other, and at that point people
+might not be interested in the device anymore if all they want to do is play MP3s. But some time after you split these
+two devices apart, you might find that the MP3 player can benefit from a different battery design or smaller hard drive
+than the video player, at which point you're no longer duplicating anything; what initially started out as duplication
+to allow this interdependent device to split into two separate, independent devices might later turn out to yield
+designs and implementations that are no longer redundant at all.
+
+It's worth considering things from the perspective of the one using a library. Would you actually want to use a library
+that minimizes code duplication? Chances are that you won't because one that does will naturally depend on other
+libraries. And those other libraries might depend on other libraries to avoid duplicating their code, and so on, until
+you might need to import/link 50 different libraries to just to get some basic functionality like loading and playing an
+audio file, and that becomes very unwieldy. Meanwhile if such an audio library deliberately chose to duplicate some
+things here and there to achieve its independence, it becomes so much easier to use in new projects, and chances are
+that it won't need to be updated nearly as often since it won't need to change as a result of one its dependent external
+libraries changing which might be trying to fulfill a much more generalized purpose than what the audio library needs.
+
+So sometimes it's worth deliberately choosing to duplicate a little bit (consciously, never out of laziness -- actually
+out of diligence) in order to decouple a library and make it independent because, through that independence, it achieves
+a wider range of practical applicability and even stability (no more afferent couplings). If you want to design the most
+reusable libraries possible that will last you from one project to the next and over the years, then on top of narrowing
+its scope to the minimum, I would actually suggest considering duplicating a little bit here. And naturally write unit
+tests and make sure it's really thoroughly tested and reliable at what it's doing. This is only for the libraries that
+you really want to take the time to generalize to a point that goes far beyond a single project."