Kim du Toit deals with someone who uses the oft-repeated logic of "it's my land isn't it?"
The answer to this is that it is, but it won't always be. Building codes protect no just the current owner from huckster contractors, but also the future owner and his neighbors. You can't replace them with home inspections, because home inspections don't include looking inside the walls at plumbing, wiring, structure, and weather stripping. Which means the state takes over and does these things for us. That doesn't mean they do it well, lots of inspectors are incompetent or, worse, petty bureaucrats. But they do it and they need to do it.
It isn't a coincidence that earliest written codes like Ur-Nammu or Hammurabi involve laws about land drainage. For that matter Deuteronomy has a law about safety railings. This is basic and important stuff.