(Un)fortunately, this week is my last with the title "Software Engineer." Before I banish the last four years of my professional life out of my mind entirely, I wanted to document some of my observations on the industry, particularly in web development. My first real programming job was at a small, series A startup in Washington, DC. We operated in the humanitarian sector, building a robust data processing and analysis application for NGOs, government, and other aid organizations. I was one of a handful of core engineers on the team, and I worked primarily on the data engineering side of the house. Since then, I've been in the biotech space working on front-end and data visualization projects at a very large company. I've been fortunate enough to work with some brilliant people, play around with a crazy number of libraries and frameworks, and experience the full range of startup insanity to corporate stuffiness. My time as a software engineer has been largely positive, and I'm grateful to be going into graduate school with these skills. To that end, here is a random assortment of observations from the industry. Of course, these notes reflect my own experience and are not indicative of the software world as a whole.
- Interviewing is exceptionally hard - and often completely removed from the day-to-day responsibilities of software engineering. FAANG (Facebook, Apple, Amazon, Netflix, Google) and adjacent companies have defined a set of standards for the industry on interviewing, often requiring candidates to know heavy computer science theory to succeed. Almost every software engineer I know has a copy of the infamous "Cracking the Coding Interview" on their bookshelf - an 800+ page manuscript detailing exactly how to land a job at one of these companies. Many bootcamps offer (expensive) courses on how to pass these interviews, and many study for months before even applying to these companies. This, in my opinion, is not a great measure of how successful one will be at their software engineering job. It discourages good candidates from applying, increases feelings of imposter syndrome for those who don't "pass", and overall creates a hyper-competitive and often-toxic environment where individuals are focused more on studying for interviews than fundamental programming or computer science paradigms. The best interviews I've had contained some combination of an at-home coding challenge (some even paid!), culture-fit interviews, and a technical presentation or discussion. In these interviews, you can highlight your strengths, talk through your experience in depth, and are not required to know obscure algorithms to make it to the next round. Perhaps I'm just salty because I got rejected from Google after failing to implement a suffix tree data structure on Google Docs.
- Regulated systems are really challenging. I've worked on FDA-regulated medical software, and in my opinion, the company for which I worked failed to reconcile the desire to move quickly and the FDA's requirements for maintaining product approval. Each code change required pages and pages of documentation, and any bug found in production meant we were risking our product's approval and the company's standing with the FDA. The company, despite this, pushed forward with an "Agile" approach to software engineering, aiming to maximize feature development in two-week cycles. To me, these are incongruous. Applying the trendiest development framework to a product which, by design, cannot "move fast" is a failure to recognize the challenges of the space you're operating in, and will cause friction between engineers and product teams.
- The barriers to entry might be lower than other fields, but there is a lot of work to be done on the diversity and inclusion front. Software engineering evangelists love to say that anyone can become a software engineer. No college degree needed: just look at Mark Zuckerberg and Bill Gates! All you need is discipline! While this may be true to an extent, the software industry needs to reckon with the fact that it is overwhelming male and white. Black and Latinx individuals are poorly represented at FAANG companies, with their percentages of representation staying largely stagnant (source: Wired). Women especially are systemically shut out of leadership roles and are often paid less for the same work. This is both morally and economically unsustainable. If the tech industry wants to attract and retain the best talent to stay competitive, it will need structural reform in addressing these deeply-rooted inequities.
- Javascript is everywhere. According to the 2020 Stackoverflow Developer Survey, web technologies (Javascript, HTML/CSS, Typescript, React, etc.) are overwhelmingly widespread and popular. While the ecosystem does have some serious flaws to contend with (look no further than the infamous left-pad incident for proof), its ubiquity has allowed for strong developer communities and fantastic open-source software. For rapid prototyping and iteration, Javascript & co. really shine. It still amazes me that I can spin up a new React application and deploy it to the world-wide-web in less than an hour. Fortunately, web frameworks are starting to reach a certain level of maturity. Advanced features in the language itself (in ES6 and beyond) have solved some issues of inconsistent behavior, overdependence on libraries, and unmaintainable code that plague earlier Javascript projects. New open-source projects continue to push the boundary of what's possible on the web. I'm personally really interested in seeing how large-scale data visualization frameworks will play out. It's a bit of a nascent field right now, but d3-powered visualization and sharing tools like Uber's Deck.gl and Observable have the potential to democratize data analysis and visualization.
- I have no idea what Kubernetes is, and every time someone tries to explain it to me, I get more confused.