Some reflections on project management

During the last year, I became a software project manager. Wow. Not something I was expecting.
It's a chaotic (to say the least) environment I've been thrown into -- especially in the realm of decision making (in fact, I think I probably need to sit down and think exactly what environment I'm in a la the Cyefin framework).
The first few months felt like a breathless attempt to keep up, but finally, this Friday, I had a chance to sit down and take baby steps toward acquiring a knowledge framework for my job. Of course, my company has some online training modules, but I wanted an individual voice of experience (and my mentor is out on leave out the moment). Of course, googling 'best practices agile software project management' is going to get me a lot of hits -- but many of those are from paid consultants, and many are highly general. I wanted something more grounded and personal than that.

Luckily, I happened to remember that a blogger whose personal writings I've been following for years, is also a (biotech) project manager & consultant! I spent about 2 hours devouring her content over at  Beyond Managing  and at Chronicle Vitae .

I love her pragmatism and honesty -- particularly her admission that she was an accidental manager -- thrown into the role because she was excellent as an individual contributor -- and that she didn't at first view project management as 'real work' compared to technical work. I identify on both counts!  Or rather, I understood even before I became one that project management was real work -- because I've seen both excellent and poor examples in the workplace --- but I didn't know if it could give me the satisfied feeling of 'getting stuff done' that say, publishing an API reference from scratch could.  Now I am starting to understand from my own experience (and from reading hers) that yes, the process of figuring out how to get work done well as a PM can present mental challenges and satisfactions on par with 'concrete' technical work. No, the output is not so tangible -- but that's OK with me. I get my reward from the process, not from work artifacts. So what parts do I like the best? So far, I most enjoy:

• digging deep to truly understand the implications of the technical work in terms of schedule, risks, and dependencies
• translating detailed technical explanations from development managers into summaries for executives
• noticing when critical conversations are lagging and driving them to conclusion
• reconciling ambiguity underlying functional specifications and turning research language into software engineering language
• being the person who has the answers to engineers' questions about priorities, cross-functional teams; and articulating agile best practices. 

So, I can definitely see how project management could give me years of interesting challenges, but a lingering question for me at this point is -- have I gotten the technical depth I wanted as an individual contributor yet?  How to balance the 'fun' of deep-diving into a new technical area with the more general approach necessary for a manager? Or is it a false dichotomy I'm constructing here? (I've heard a dev exec mention that he thinks the best project managers are those who are most technical, after all).

End-to-end development of Swagger API docs, including Dockerization

I've known how to edit API docs in swagger for about a year or so, but as a solo writer recently I took taught myself the end-to-end process. Since I don't know any other information developers who've done the process end-to-end at my company, I taught myself the technology and occasionally asked my devs for help. Here's what I did, working on and off over the course of a few months:

-researched pros and cons of swagger annotations in the java spec itself or in a separate spec file. Went with the latter, even though at first I felt this violated single source of truth principles. (the build process was so precarious and difficult for in-java annotations that it wasn't worth my time)

-read the java code in order to author the yaml spec by hand. API handyman came to my rescue here: https://apihandyman.io/writing-openapi-swagger-specification-tutorial-part-1-introduction/

-Installed swagger-editor to get instant feedback on my edits.

-Created a web app to publish my docs using Tomcat.  Made some use of https://idratherbewriting.com/learnapidoc/pubapis_swagger.html

-Got feedback that I actually needed to publish my docs using Websphere Liberty Profile instead of Tomcat.  Created a WLP web app using this tutorial: https://developer.ibm.com/wasdev/blog/2016/02/17/exposing-liberty-rest-apis-swagger/. In theory this would have been easy, but the most difficult part was finding the right version of WLP to install on Eclipse marketplace; the versioning conventions just seemed really confusing to me.

-Got told I needed to put my web app in Docker. Fortunately, there were a couple other Dockerized web apps in our team I could use as an example, plus the Docker intro videos are really good (https://training.docker.com/introduction-to-docker). I successfully Dockerized my WLP app, and handed it off to a developer.

YAY!

Some new roles

I've taken on some new roles in my team recently:

Agile coach

I'll be the agile coach for a 3-month stint for my team.  We face BIG challenges in implementing agile, the biggest being:

-lack of tooling:  since we work in highly regulated industry, our tools are waterfall-focused, and jumping over to agile-focused tools is a huge approval process. We hope (*HOPE!*) to have better tools -- ie GitHub, Jira support for Agile -- in place by late 2017.

-team complexity -- Coordination across executives, researchers, offering management, sw dev teams, ux, etc, across multiple offerings across TWO companies (parent company/acquired company) that both have required, differing regulatory processes? Very very tough. Biggest challenge I see is at the moment, some upcoming projects lack epics/hills. My hope is that for the projects that DO have epics/hills, we can build out our stories/tasks so as to provide workable examples/templates for other projects.

In the meantime the program manager and I have cobbled together an awkward issue tracking process involving issue types, links, titles, and labels.  It's incredibly manual at this point. We'll hobble along until later in the year, when better Agile support will be available, and I'll be checking regularly with my scrum masters to see if we can make any improvements given the limitations.

Software Release Manager
At my company, there's a traditional documentation set that technical writers are responsible for,  and another set that developers are traditionally responsible for. But I have enough technical depth that I can contribute to the developer docs too (stuff like functional specs, SW design specs, etc), and so I am.  I've also taken over project managing these docs from my SW development boss.

This also means I'm working on defining documentation processes, since our docs are all highly interrelated and highly regulated. For example, helping to come up with a process for tracking and maintaining our third-party components in a 400-row-plus spreadsheet (that one was a bear!); defining and writing up in our wiki how to document unit tests and code reviews, etc, etc.

Building the tools I document

I've come a long way as a self-taught developer. My software dev team is shorthanded at the moment, and the lead architect pinged me recently and said something along the lines of, "I know it's not in your job description, but you seem to pick up on stuff pretty quickly; want to make some deployment tools for us? If so, read up on ansible and the python jinja2 library."

And guess what? I'm getting it done! He gave the specs for what the tool should do*, I worked independently for the most part, occasionally checking in to make sure I was interpreting his specs correctly, read up on ansible, kubernetes, a few python modules (jinja2, argparse, and configparser), learned a lot about linux (our deployment environment), a few features of yaml and..... two weeks later I have a working script that is intended to be used by our devops team in production. The experience was such a contrast to my early Python days, when error messages were mysterious and frustrating; most of the time, I had an instinct for what was wrong and quickly debugged.

So at the moment, I don't have to ask, "when will the tool be ready to document, and what will it do?" Because I'm writing it. Which is a good thing, because the tool I'm coding will probably form about 30% of the documentation in the install/config guide I'm writing for our product.




*In its current form, the tool takes some yaml specs for kubernetes deployments, injects some configuration values into them using jinja2 (such as dynamic IPs for kubernetes services), copies those deployment specs to remote hosts using ansible, and deploys them via kubernetes' kubectl command-line interface.

Studying machine learning through Coursera

Since I document tech that involves machine learning and natural language processing, I've been meaning for a while to take Coursera's Machine Learning course. And I have a bit of bandwidth at work now, so that's what I'm doing.

It's great! I'll admit, I was doubtful if I'd understand anything since (shameful secret!) I never took calculus in high school; I opted for AP stats instead. At the time I was totally focused on studying classical music -- it never occurred to me I might wish one day I had.

Fortunately, the instructor is really, really good at going slow and reassuringly pointing out when we don't actually have to understand the calculus or stats he's talking about.  He's also really good a giving us an intuitive understanding of the purpose of a formula -- so, while I don't understand how to calculate a partial derivative for gradient descent, I do understand how taking the tangent of a curve gets us to take a step in the direction of steepest descent.

It also helps to have a spouse with a PhD in math; when the instructor mentioned the "line search" strategy for choosing step size but didn't go into details, my husband sketched out a 2-D representation that made intuitive sense to me.

I thought doing the homework would force me to understand the calculus/stats the teacher said we didn't need to understand...but it turns out that's not the case. Instead, so far it's more about learning the ins and outs of matrix math and of Octave, the programming language we're using. I'll admit, I'd be pretty lost without StackOverflow and the course wiki -- but when I see students posting the same questions I had, it reassures me that I'm somewhat on the right track.

This course got me thinking about how being totally unintimidated and getting by with "sorta understanding" is a valuable tech writer skill. Back in my student days, I wanted to master everything from the get-go. But teaching myself coding and learning about technology on the job taught me to grab at the big picture from the get-go instead, and fill in the confusing bits as I go. It's like my ability to read a book or a bunch of online info really quickly and connect the big-picture points first (something my engineering-trained spouse would never do).  That's not an approach that works for every field, but it sure works for a tech writer!

Another thing this course got me thinking about was how I love to learn. Again, back in my student days, there was so much to learn that I easily prioritized my strengths over less interesting material. Sure, math was OK, but there were plenty of more compelling subjects to study.  But as adults, we have less chance to learn new stuff (again, another reason I like being a tech writer -- I get to study!), and I find myself revisiting math with fresh appreciation. Maybe math is not my main talent, but who cares? I'm learning, and I love how my brain feels when that happens.

update
In the end, I did the homework and implemented gradient descent for linear regression and for logistic regression in Octave (with much online help). After that, I mostly audited the course and took notes. Some highlights I took away (probably I've mis-stated a bunch of stuff; I'd have to rewatch the videos to get it exactly right!):

• neural networks let you compute truth tables for logic operations like AND, OR, XNOR, bv varying the weights (thetas) of your inputs and then running the sigmoid function on the sum of the weighted inputs. In each hidden layer, you learn a new theta through forward propagation, i.e. through reapplying the sigmoid function(???)
• support vector machines are similar to logistic regression, with the addition of kernels (landmarks to which you measure the similarity, or distance, of your examples). In cost optimization, you compute the 'vector inner product'  (just a Pythagorean measure of length) between theta and training example, and try to maximize that distance, p, in order to get large margins between the examples and the decision boundary
• Unsupervised learning using k-means clustering was conceptually a lot simpler than I expected. You randomly initialize some k number of cluster (c) centroids (mu)  from your training set, compute the distances of the training points to the centroids, and classify them into clusters based on how far they are. Then you take the means of the distances in each cluster, place the centroids at those means, and recalculate the distances in order to come up with new clusters, then retake the means..etc.

My first REST API documentation

I've documented low-level hardware APIs, but never REST APIs -- I learned a ton working in Swagger for a Watson service API.

Looking back, it breaks down into two parts:

Learning about APIs, and learning about the Watson service.

Learning about REST APIs:

·        doc tutorials - For an introduction to fundamental API concepts and terminology, I found Tom Johnson's tutorial on documenting APIs invaluable-- he even had a post on Swagger specifically! (Tom's a prominent blogger in tech communications; in fact, I read his blog really intensively about 5 years ago when I decided to go into tech writing!) He helped me talk intelligently about endpoints, resources, query strings, etc.

·        building swaggger - Getting a local swagger build up and running was a task and a half due to the build structure (which has since been simplified, thankfully!). I leaned heavily on colleagues, but I was eventually able to get the build running on my Linux virtual machine -- though not, strangely, only my local Windows machine.  I must have spent days on this, yikes! But I wrote or updated 3 process docs on the subject, so hopefully the next person will have less pain.

·        modifying source code - Adding annotations into the source java files using @API annotations was not too difficult. But I ran into some bugs, and for the first time, posted and got answers about bugs on a developer forum(https://groups.google.com/forum/?fromgroups#!topic/swagger-swaggersocket/Xq7jCJS7WRc, https://groups.google.com/forum/?fromgroups#!topic/swagger-swaggersocket/ZCg4UZYYn08 ).

·        REST fundamentals - Learning to play around with the API forced me to learn some REST fundamentals too--when can I enter something in a parameter, versus typing a query string in the request URL? What syntax to use to  get parameter fields into the URL query strings? How can I discover whether a parameter field takes JSON or some other syntax? (answer--I think--is that there's not discoverable method; that's why you need the docs!)

Learning about the Watson service

My understanding of the resources I'm interacting with evolved quite substantially, though examining the source files for schemas and instance data, playing with the API, and most importantly, talking to the developers. I won't go into the details since the service isn't public yet, but I learned a great deal about type systems and entity/relationship modeling schemas.

L

Good news (and bad news for this blog)

The good news is, I got a new job that I really like! I'm now working as an information developer for IBM Watson; the technology is really interesting.

The bad news for this blog is, we're encouraged to blog internally, which affects our performance reviews. So, now I'm blogging about my job at my job, and I doubt I'll keep up two separate blogs. That said, I might find time here and again to adapt one of my internal posts to a public audience -- we'll see!