Morally Sound AI

Notes on building AI responsibly

AI in Software Engineering

This one is personal. I have been writing software for a long time, and the last couple of years have changed how I do it more than the previous ten. AI now touches most of the lifecycle – requirements, design, code, tests, deploys, and the incident at 2am. The pitch is that it takes the repetitive work off your plate so you can think about the actual problem. That has mostly been my experience. It also comes with some fine print.

Code assistants are the obvious example. Autocomplete that finishes the function, boilerplate on demand, whole features from a paragraph of English. My output is up and I would not go back. But generated code does not know your security posture or your architecture, and it will happily produce something plausible and wrong. The bigger risk is the slow one: engineers who stop reading what gets generated, or who lose the feel for the code underneath. The tool is only as safe as the review it gets.

Beyond code generation there is bug detection, vulnerability scanning, performance tuning, and predicting where a system will fall over next. Useful, all of it. The questions it drags in are less tidy. Who owns code a model wrote? Who is accountable when a model-introduced bug takes down production? And which parts of this job stay human once the typing is automated? My guess is judgment, taste, and knowing what not to build. Nobody knows for sure yet.

Working through that needs more than engineers. Security people, lawyers who understand IP, the researchers building the models, and someone thinking about ethics who is not also trying to hit a ship date. Without that, the norms for how humans and models work together get set by whoever moves fastest, and that is rarely the person thinking about consequences.

The version I want is one where these tools make engineers more ambitious, not just faster. Used purely to cut headcount or shave sprint time, they will produce more software and worse software. Used with real review and a culture that still values understanding the code, they let a small team build things that used to need a large one. I am betting on the second version. Ask me again in a year.