Developing a game engine is an insanely complex task. Not only will you spend thousands of hours trying to get the first version of your engine to work, but it will also test your persistence. Thus, developing a game engine is more of a psychological feat than a technical one.
Before you embark on this journey, I have some tips for you.
Play with other engines
Before developing your engine, spend some time developing a couple of games and using a couple of game engines. Use this experience to gather features that you like and dislike from each engine. For example, you may like how friendly an engine's API is versus the other.
Keep this in mind: Developing an engine does not only mean implementing amazing graphical features. It also means developing a user-friendly API which developers love.
View it as your favorite hobby
Again, developing a game engine is insanely difficult. I can't even express how difficult it is for one person to develop one. And not everyone has the stomach to work for thousands of hours before seeing their creation come to life.
So, if you want to see this through, you need to view engine development as a hobby. Don't rush, just sit back, open your laptop and enjoy. It may take you one year, two years, five years, who cares. Just enjoy and fall in love with your engine.
Is OK if some features are half baked
Not every feature in your engine needs to work correctly. Not in the beginning anyway. If rendering Shadows work 70% of the time but fail within some conditions; make a note of it, take a break and move on.
Don't kill yourself over it. Remember, this is your hobby, your favorite pastime. You have no deadlines, no nothing. All you need to do is promise yourself that after a while you will come back to fix the issue.
It is important to consider that when you start the journey to develop your engine, you will not know everything about computer graphics. And you will not know what you don't know.
Features may not work as you hoped for initially, but this is OK. It is a process of failing and learning. Let me give you a recent example.
Last week I ported the engine from OpenGL to Metal. As I was doing the porting, I noticed that some implementations I had done a couple of years ago were incorrect. At the time, I thought that they were correct. But the experience that I've gained over the years spot these mistakes right away.
Don't let perfection immobilize you. If a feature is not working 100% as you intended, make a note of it, take a break and make a promise that you will come back to fix it in the near future.
Thanks for reading