For example, a schema is an abstraction. Choosing to have a strictly defined schema for your stored data is choosing to add a layer of abstraction. You could say simply store things as JSON, directly, serialize whatever object you have into JSON and be done with it.
Or you could choose to add a layer of validation and create a JSON Schema. Then set things up so the concrete data is created using the abstract schema definition in a way that also automatically set ups validation of the data using that schema.
Now sometimes this is overboard and too complicated for whatever you're doing, sometimes this is an amazing addition to a code base that really simplifies and make you more productive.
Edit: I'll give another more simple example as well, to show how abstractions are relevant at all levels.
Take a Player Class, where Player has a position in the world map.
You could go the concrete direct route:
Player {
String name;
List<item> inventory;
int health;
int x; // x position in world
int y; // y position in world
}
Player {
String name;
List<item> inventory;
int health;
Position position;
}
Position {
int x;
int y;
}
And there's scenarios where it wouldn't benefit much.
The solution where position is just concrete ints on the existing Player class is more concrete and direct, but not always the best.
Now a mistake I find mid-level engineers make is they'll read a blog or book saying it's much better to abstract out Position like this for x, y, z reasons. And then they'll do it for everything, they'll apply it to `name` for example:
Player {
PlayerName name;
...
}
PlayerName {
String name;
}
You could conclude never to abstract anything ever again, and that would be better than the monstrous over-abstracted everything for no reason and often badly implemented at that, and that's an improvement, later on you'll get even better and learn the nuances, when, why and what abstractions in just the right place, the right amount, in just the right way, and it'll be even better.
The focus on building abstractions is misguided. You don't build an abstraction because you have stuff lying around that implements things -- you build an abstraction because you need it to do your job. That's the only valid reason to ever build an abstraction: you, as the consumer, need the abstraction to do (or to define) your own job. As a consequence of this, most abstractions should be defined before they're implemented. It really feels like most people miss the point on this one, and that's why we end up with bloated abstractions. They're not about what you have. They're about what you need.
That means you should actually have lots of abstractions (assuming you have lots of different needs throughout your code), and they should all be simple, small, and clear. It should be obvious how to implement them, and obvious what they're used for. They have to be: that's how they were built to begin with.
(In fact, while we're at it, the focus on classes is misguided too. Why does everyone think you need to make classes that mirror common nouns in real life? Bad CS education?)
I could absolutely see "Position" (and, critically, everything in it) as something some service needs to do its job. In fact by simply looking at that class, I've learned a lot about how your game works: it's 2D (no Z) and probably tile-based (ints, not floats). We've made a decision: that's how position works in this game. How does movement work? Start that next -- it will use Position. Keep picking away at the edges, making useful decisions about the game, etc. Build abstractions only when you need them to answer that question: "how does X work in this game?" You will never get to the point where you build a "Player" class like that, which is why I can confidently say that a codebase with such a class must inevitably suck.
And then there is Unreal Engine 5's ACharacter class[0] :-P. I recommend checking the superclasses too.
[0] https://docs.unrealengine.com/5.0/en-US/API/Runtime/Engine/G...
> Characters are Pawns [AI or human decision-maker] that have a mesh, collision, and built-in movement logic.
Indeed that's a combination of a lot of different responsibilities. Probably too many. Why built-in visuals (mesh) but not built-in audio? Why a mesh and not built-in particle effects? I'm guessing it's just because that is the combination that they found helps inexperienced coders make cool things quickly. I'd be really curious whether people who spend a lot of time tweaking their engine, or make games that are more complicated than just Another FPS, actually use that class much. I suspect they either don't, or they have several similar varieties of their own, which they sorta switch between as it makes sense and then go "Dammit, I wish we had made this an ACharacterTypeSeven, not an ACharacterTypeSix!!"
Of course there are times when you combine responsibilities together into larger objects, but the trick there is to always accept that this is just one projection, one perspective on the entity. If you start to think of that ACharacter object as the character, you'll have problems. It's an arbitrary boundary. When you come up with a cool idea to, say, have your character split in two parts with independent motion before merging back together a few seconds later, is that two ACharacter instances or one? You've duplicated some parts of it, but not others.
"But dude, YAGNI! Don't try to predict the future" you say, missing the point. I'm not saying restructure your code just in case someone wants to split characters in two later -- that's YAGNI. I'm saying throw what-ifs at your code to see if it holds together as a sensible concept right now. You future-proof your code by making sure its concepts are clean, independent, and composable, not by trying to predict the future. My character-splitting example is not an example of something we should plan for, but rather an example of why the concepts may not actually fit together that well. When I look at ACharacter, I don't see something that's composable -- I see something that's already composed for you, and if you want a different composition, it looks like a pain in the ass. That tradeoff makes sense if your main goal is to help inexperienced coders make cool things quickly, but it does not make sense for the codebase you rolled yourself.
It actually all starts to feel like.... SQL! State is stored globally in a defined schema and queried as needed by the system.
But you can't do this if your compositions are preordained from on high by a rigid class hierarchy, the data is crystallized into the "blessed" projection and that's that. It's analogous to your SQL queries being constrained to solely static views. No JOIN. No GROUP BY. No WHERE.
Actually i'd say the opposite, "Unreal does it" indeed doesn't mean it's the right choice, but that "Unreal does it" proves that in practice that stuff doesn't matter - Unreal is a codebase going back decades and yet it is as popular among developers as it ever was (some developers even throw away their own engines to switch to it).
So while these topics can be amusing to read, in reality they are bikeshedding of little more importance than using spaces vs tabs or where to put curly braces and how that affects diff tools.
> I'm guessing it's just because that is the combination that they found helps inexperienced coders make cool things quickly
Is not "making cool things quickly" the essence of enterprise programming? Sure, you can make the cleanest, most perfectly abstracted code for yourself when the requirements are well-defined and unchanging, but that's not the environment you find in business. One might contend that such a combination is the optimum for enterprise programming/making cool things quickly.
I think it's not. I'm not sure there's anything quick about enterprise programming. If you're cynical, the essence of enterprise programming is selling absurdly expensive software to clueless senior leadership that will never use it. If you're optimistic, the essence of enterprise programming is being a good data steward while elegantly handling the needs of a lot of different stakeholders and interfacing with a lot of different systems (some automated, some implemented only in brains).
In game programming, if you can't figure out a good way to get the camera to work in one particular level, you just scrap or redesign the level. In enterprise programming, if you can't figure out a way to import a particular Excel format, you could seriously harm the usefulness of your project or even lose a contract. You have to "get it done", and there are a lot of "its" to get done.
When I say "make cool things quickly", I mean that there are tradeoffs between having high velocity in the beginning (standard templates, pre-defined assets, content management systems, implement the whole thing in Salesforce) vs. maintaining that velocity through the lifecycle of a potentially very long project. I claim that one of the things that makes popular frameworks popular is because they tend to heavily prioritize the former over the latter. That is great for going from 0 code to shipped quickly, but it's the wrong choice for 5+ year projects like you see in enterprise.
In fact I think one of the (many) things that poisons modern enterprise programming is the emphasis on tools that get you going quickly, rather than tools that stay loyally by your side through the whole project lifecycle. MongoDB is quick and easy to set up, because you can just throw whatever JSON objects you want in there, without spending all that time worrying about "schema" (I do think people spend too long worrying about schema, but the answer is not to abandon it -- that's a whole other subject). But you still have a schema! It's just that now you don't have a dedicated tool to help you with it, and as your needs and data change, you're the one responsible for keeping it up to date. It seems very easy to get mismatched or out of date JSON objects in there and very hard to clean it up (although I'm no expert on JSON databases). Whereas SQL Server or PostgreSQL will support your changing schema very well throughout the whole project lifecycle. If you took over a 15-year-old project, would you rather it had been using Postgres or MongoDB that whole time? I know I'd prefer Postgres.
I've no experience with game dev, but in other areas of development what you need is often not known ahead of time (which I believe the parent is trying to say). Operating under those conditions makes the Position abstraction somewhat arbitrary (until it's obvious it's needed by other parts of the system). Aggressive refactoring and robust testing are necessary when operating under these conditions.
Well that's kind of the point of software engineering, isn't it? Actually typing code is only a small part of software engineering, and if you don't know what you need yet, then you're probably not ready to write code. That doesn't mean you're not being a productive software engineer! It just means you're still working toward that point.
I could be more clear about "what you need" actually means. Let's say I sit down to (A) prove the four-color theorem, which says that no more than 4 colors are needed to color in a 2D map with no two adjacent regions having the same color; (B) write a function to color a map using as few colors as possible. Before I can start on the meat of it, I have to decide exactly what I mean by "map". Anything that (A) relies on my proof or (B) calls my function is going to need to turn their data into the kind of "map" I'm working with.
Oh, I know what a map is: it's an ArcGIS Pro 10.7 Geodatabase with a Polygon layer! Hand me one of those and I'll assign a 32-bit ARGB value to each polygon. What? You don't have ArcGIS Pro 10.7? Well, sucks to be you. Obviously it has to be that, since I need insert esoteric proprietary feature in order to color it.
Hmm, well, okay, maybe I don't need every feature of ArcGIS Pro 10.7 Geodatabase Polygon layers. In fact, I really just need a list of regions and how they're connected. Do I need literally every wiggle-waggle of every border between each region? Well, not really ... in fact all I really need to know is which ones touch, and which ones don't. In fact, it turns out what I need is a Planar Graph. It probably shouldn't have any loops (nodes connected to themselves) either. A loopless planar graph. If you hand me one of those, I can color it for you -- in fact, I'll just assign each node 1 through (up to) 4 to represent the colors, and you can do whatever you want with that information, rather than me picking the actual ARGB values for you.
The reason I associated it with a math proof is because it's more clear in that case that reducing the preconditions on the objects you accept increases the power of your proof. Proving something interesting about all multiples of 5 is less powerful than proving it about all integers, and less powerful than proving it about all elements of an Abelian Group, etc. Writing your code to work on any loopless planar graph is much more powerful than writing it to only work on <Random Complex Proprietary Format>.
And we settled on "loopless planar graph" not because we had a bunch of graphs lying around -- we probably didn't! We probably actually had a map representation we've used elsewhere. We settled on "loopless planar graph" because that is the minimal possible description of the objects we can run our code on. That's what I mean by "what we need to do our job". That is the birth of an abstraction.
One of the mistakes I really hate seeing people do in typed languages is not using types for distinctly important data sets - a good example is when you have ciphertext and plaintext being passed around. At an application level you want to be really sure that you're going to be accepting and using ciphertext in the parts that need it - even if they're both technically valid string types.
Meh, I'd say strictly defined schema, moving database consistency logic to DB etc. is an example of a bad abstraction in most cases I've seen it used. The idea sounded really good when I was a junior, you can have data layer enforce integrity from all sources.
Except most applications are exclusive owner of the DB and it's schema - even in the microservice world it's one database per service. If I see other apps hooked up it's passive readers/exporting/logging/etc.
SQL databases still don't play well with being in sync with the repo (it requires specialized tools or extra care, which again usually means extra tools).
Database schema constraints are often crude and/or complex and don't scale well - it's common for people to avoid even rudimentary things like foreign keys because of what it can mean in terms of locking/ordering and write throughput. And using things like callbacks etc. good luck.
But from what I've seen using schema to enforce data consistency brings more problems than benefits.
An abstraction has no material presence, so in code, concretions are the actual data in memory in their exact place and precise arrangements and linking and all that on a precise machine.
Source code is an abstract representation of a running program for example. Everything in source code form is an abstraction.
Generally an abstraction has gaps, it can't actually be run into the exact behavior you want on its own.
A schema is an abstraction, because schema isn't executable into the concrete running behavior you're trying to implement. A schema or a data specification (which is just another word for schema), is definitely an abstraction. It abstracts over the actual concrete data you will have at runtime.
Everything that is not in the final concrete running form it needs to be at runtime is an abstraction.
A specification abstracts the implementation away, it's an abstract idea of what you want, but does not specify the implementation.
Unfortunately the concept of abstraction is itself very abstract, but one thing a lot of people don't realize is how many things are actually abstractions.
Even a simple function is an abstraction. It can only run once provided actual values to its arguments, it is but a template abstracting the idea of mapping inputs to outputs using some logic. You need an instance of it with actual values for its arguments to be able to run it, a function with concrete values provided as arguments is now a concrete thing, but the function definition, i.e, the code for it is simply an abstraction.
You can further abstract over abstractions, a function signature abstract further the implementation away, to be filled in later.
And generally speaking, even a running program is simply an abstraction of reality, a simulation of something real, but this is when you start to enter product design, how to best abstract over the real life use cases you're trying to have a program that can represent, model and simulate.
Programming Languages and other frameworks often provide you means of modeling abstractions, constructs that can help you define your own abstractions. Those tools will vary from language to language, in some OO language like I said you're given Classes, static types, Interfaces, inheritance hierarchies, etc. In some functional languages you'll be given abstract data types, functions, higher order functions, type classes, etc.
I could keep going on, but I find the concept of abstraction itself is often misunderstood.
P.S.: You can easily argue a different definition, arguing semantics has no definite truth, definitions of words are just axioms we define. I believe it is more useful and beneficial to define abstraction as I just did for being able to better reason about and make judgement calls as to how exactly to structure and design software code. I'd encourage others to give it a try, attempt to rediscover abstractions how I just described, and you'll learn a lot in my experience, and you might become a better programmer out of it.
Just my 2 cents.