There are no strict rules in software development
Take your skills to the next level!
The Persistence Hub is the place to be for every Java developer. It gives you access to all my premium video courses, monthly Java Persistence News, monthly coding problems, and regular expert sessions.
A few days ago, I joined a discussion on twitter, that started with a statement, that you should not expose your entities as a REST endpoint except you are creating an application for a stage demo. The reason why I joined that discussion was, that I completely disagreed with that statement and still do! I write this post to explain why I do not agree with this statement and any other statements that are like always do this or never do that.
Don’t follow strict rules
The main reason why you should NOT follow these kinds of statements is, that they completely ignore the specific problem that should be solved by a specific piece of software. Solving problems or pain points for the user is the main reason why we implement software (OK and sometimes because it’s fun). If we want to do that in an efficient way, we should use all the options that we have. And that are a lot, if we do not limit them ourselves by ignoring a huge part of them.
Build solutions
I am often wondering why people think you should solve every problem in the same way. The main reasons I have read and heard so far are:
- Unification of the code base provides guidance to the developers and makes it easier to switch between different parts of a project or even between hole projects.
- Applying patterns and best practices helps to build high quality software.
As you might guess, I do not agree with that. Applying always the same structure or technology to implement a new service or use case can really help you to get started with new implementations or to find your way around in a new project. But technology or code structure are normally not the main issue, if you are new to a project. Business requirements, workflows and how the user thinks are the difficult thinks to understand. And these are not addressed by these kinds of unification. To make it even worse, unification often hides the real intention of the code because the developer forced the logic into a specific structure or technology.
The advocates of the current microservice hype take this into account and proclaim that you should select one specific technology stack for each service. I am not sure, if this is always a good idea or if it is a little too extreme. But it might trigger the required process to think about the requirements and how they can be solved in the best way. That can be the same way you used in all the other projects or something completely new, as long as the decision is based on the requirements.
Software architects often like to justify a uniform code structure or technology choice for all parts of an application by using existing patterns or best practices. This is completely against the idea of best practices and patterns and does more harm than it provides benefits. Don’t get me wrong, applying patterns and best practices are a good idea and can bring huge benefits. But each of them was defined for a specific kind of problem. There is no pattern or architecture to rule them all! So please, before you apply a pattern or architecture, make sure to read about it and understand its intention. Using a pattern under the wrong conditions can create lots of problems.
How to do it
The most applications consist of more or less 3 different parts that provide very different challenges:
- simple CRUD services that provide nearly no challenges at all,
- complex business logic and
- data oriented reporting with complex queries.
Should you handle these 3 parts in the same way? No, there is no reason for it! Understand the challenges of each part and choose technologies and patterns that help you to solve them!
Here is how I like to approach the 3 parts, but keep in mind, that these are only general recommendations. You might have different requirements that require different choices.
Productivity is the most import thing, if you need to implement a simple (and often boring) CRUD service. So keep it as simple as possible. There is no need to apply a multi-tier architecture for your business tier or to use some fancy patterns, if you only need to store some data in the database. In this case it can be perfectly fine to expose an entity as a REST endpoint and add some bean validation annotations to it. Just make sure that you have a direct mapping between the JSON input and your entity and that you want to apply the same validation rules on both of them. If you need to support a different data structure or a different set of validation rules, value objects might be the better approach.
The business logic part of your application contains the most source code and should also provide the most freedom to implement a good solution. Depending on the specific requirements you can expose entities or specific value objects or both as REST endpoints and apply the validation that is required to them. Do whatever helps you to provide a good solution. Applying patterns to solve the requirements often provides the most benefits in this part of the application. But again, it is very important that the applied pattern fits to your requirements. So take your time to understand the requirements and learn about the pattern.
In general, the database or whatever datastore you use is the best part of your application to handle complex queries. Therefore you should use all the features that it can offer to query and prepare the data for your reporting use cases and keep the business tier as small as possible. If that means that you cannot use some features of the frameworks you use in your business tier, e.g. you have to use native SQL queries instead of JPQL, than do it! You introduced the database to your system architecture because it is especially good at handling data. Use it like that!
Conclusion
OK, that has become way more text than I wanted to write on this topic. The main thing you should remember is, that there are no strict rules in software development that you need to follow every time. Each and every use case is different and you should adapt your solutions to it. That does not mean that you should ignore rules, patterns or best practices in general but you need to think before you apply them. Otherwise you will create an overly complex or very messy application and both will have a negative effect on your productivity and application performance.
In general I would also emphasize a different aspect: I would even say that constraining people to do things always in a well defined and (more or less) strictly defined manner leads to much more severe consequences. It impacts directly your culture, the productivity of your engineers and especially it prevents innovation to happen. I have seen organizations where architecture teams, lead by people who never programmed themselves, defined how applications are to be build. After 10 years+ in such a regime, the developer’s know how regarding new technologies is at a legendary minimum. Many developers accommodated with the solution and stopped looking for better ways to do things. The ones that were more hungry left the company mostly. And acquiring new talents is basically impossible, or they will leave again after 1-2 years.
Given that it is obvious that costs for development and maintenance will increase drastically.
Summarizing discussing the solutions that may match the requirements is not only crucial for finding the best solution for a single problem, but also the only way to build up a positive and “agile” community that is able to keep up with the technical challenges in the future…
There is neighter much freedom nor place for fancy patterns after choosing a framework. Just choose well known stack and let it do job for you.
“…you need to think before you apply..” -> YES! YES! YES!
There is a place and time for everything, also in programming. So i agree with you, that you should not always blindly follow this or that pattern or force use certain designs.
However, I think this guideline was perhaps intended for less experienced programmers. In their case it is probably a good thing to stick to certain rules, otherwise they could go wild and create some really weird code.
OK, that can be a reason.
But the main question is, how should the less experienced programmers learn how to do it right? They not learn a lot by following some rules. Code reviews and discussions with experienced programmers are a better approach.
Thanks for your comment,
Thorben
Well done Thorben Janssen.
Firstly, I think this is a kind of over reaction to my tweet. And most importantly you ignored the main point I was trying to say and highlighting what you interpreted from my tweet.
Obviously it seems I didn’t properly put my point in 140 chars. So here I would like say it clearly what I mean by that statement.
My tweet: “We should never expose DB entities as REST endpoints unless it is a TodoList app stage demo.”
Here my main focus is “Not to expose DB entities as REST endpoints” but you seems to be completely focused on “NEVER” part.
There is no “Silver bullet”, there is no “One size fits all” and there is no “Do this way ALWAYS” or “NEVER do this way” in Software Development. I know it and I follow that all the times.
Now coming back to the point I was trying to convey “Exposing DB (JPA) entities directly as REST endpoints leads to mess”, here is why I said that.
For simple apps where you have maximum of dozen or so entities and all you need is basic CRUD operations then just go with Spring Data REST or JPARS and expose your Entities as REST services and you are done.
But if your app is not just mere CRUD operations on top of database tables and you have some complex business processes then “Directly exposing your DB entities as REST endpoints is bad idea”.
Why is it a bad idea? let me explain.
Suppose we have JPA entities with bi-directional relationships. Then we will encounter the following pain points if we directly expose entities as REST endpoints:
1. If the JPA entity is associated with Session/EntityManager then it may go into circular reference which results in StackOverflow exception.
If the entity is detached from Session then you will get famous LazyLoadingException.
2. If you try to fix the above mentioned problem by using Jackson annotations like @JsonIgnore/@JsonIgnoreProperties etc then at first sight it looks fine.
Then problems arises when you have to expose the same entity with child collections in one case and without child collections in another case.
3. If you don’t want to expose some sensitive data like passwords, creditcard details etc then you may think of putting @JsonIgnore for those properties.
But this will result in problems while you try to post the same sensitive fields because the binding process may ignore those fields as it see @JsonIgnore.
4. Now think of performing validations for the same entity in different ways in different screens.
For example to create a new entity the auto-generated primary key is not mandatory. But while updating the same entity primary key value is mandatory.
In order to put these kind of validations we start adding bean validation groups annotations.
Now it is the time to take a step back and look at the code and tell me how clean your code is and this is just beginning of the mess that we are going to create!!
Let me re-iterate my point. It’s not you, even I will argue for hours and hours why we should NOT stick to one approach for all the cases. We should definitely choose our approach based on the requirement we have in our hand.
Please don’t respond to a tweet with a lengthy blog entry without confirming “what you understood is really what the tweet author is really trying to say”.
So here, I still stand on my words. Don’t expose your database JPA entites directly as REST endpoints unless it is a toy application or a CRUD wrapper on your database tables.
I would like to appreciate your efforts putting weekly new letter which really helps me to find nice blog articles and your articles on JPA stuff.
Looking for much more useful stuff on your blog.
Cheers,
Siva.
If I understand you correctly the conflict of this discussion is more on the technical issues you get when exposing them directly, whereas if useful from a requirements perspective, exposing them in a managed and controlled way is still valid. Of course, looking at the definition of REST, exposing “Resources”, exposing database entities in many cases is quite natural and can be even (partially) automated. Though I agree, that especially the handling of relationships should be typically done quite differently (somehow more “loosely coupled”, e.g. using hyperlinks) than JPA does…
Great reply Siva! I have re-posted it to a reddit thread that picked up this conversation: http://www.reddit.com/r/java/comments/399lrc/there_are_no_strict_rules_in_software_development/cs1wenx
Hi Siva,
thanks for your long reply.
You mentioned a lot of points we agree in, especially the technical requirements you mentioned in the list. If we have at least some of these requirements in the specific use case, it makes perfect sense to not expose the entity as a REST resource. We would create a huge mess, if we would try to put two different objects into the same entity.
My main point is, that we need to identify the requirements (technical and business) before we make such a decision and should do this for every use case. Not all entities are complex or contain sensitive data and not every use case updates the relationships of an entity. So think about every use case and decide how to implement it.
Again, thanks for your long comment and sorry, if you felt offended by this post. That was not intended.
Regards,
Thorben
Because: https://www.youtube.com/watch?v=sqcLjcSloXs
Well, indeed, how could I forgot 😉
Haha, ok now I understand.
There is a strict rule, which you state in your last paragraph: Solve your own problem and don’t let frameworks tell you what your problem is.
OK, that’s a strict rule I can agree with.
Thanks for your comment!
Regards,
Thorben
I totally agree with you. What’s the downside of exposing your entities as REST endpoints? Why do I need to introduce another abstraction layer over the entities? What’s the benefict? Althought I can see the downside: more code to write and maintain. CRUD applications should be easy to build. Correctly designing your domain layer to match your json input and matching both of the will allow you to build a very concise CRUD app and in the end you’ll be greatful about your own code.
Some people tend to believe that writing simple code is wrong, but that’s completely wrong. Why make a mess when is not necessary?
Hi Mario,
thanks for your comment.
I don’t think that developers intend to write complex code. It’s more often the unintended result of a good intention 😉
Regards,
Thorben