You may be able to but I would not recommend it.
See, developing a game engine requires you to implement several algorithms such as stacks, queues, binary search trees, heaps, etc. It also requires you to implement several design patterns such as Singleton, Strategy, Observer, Composite, Model-View-Controller design pattern, etc.
These algorithms and design patterns can be implemented quite easily using C++ or in any other Object Oriented programming language.
Furthermore, a good game engine design should be modular, flexible, adaptable and maintainable. These qualities can be implemented quite easily with Object Oriented Languages such as C++, Java, etc. I don't really know how these qualities could be implemented using assembly language.
Who will be the users of your game engine? how are they going to interact with the game engine API? Most game developers prefer to use C++, Java, Objective-C, Python, etc. I have yet to meet a game developer that uses assembly to develop games.
Also, how are you planning to communicate with the OpenGL API? Even though the OpenGL API is language independent. I have not seen any examples using assembly language.
I would recommend you to develop the game engine in C++ or in any other object oriented programming language and not in assembly. If you are concerned for speed, C++ will do the trick.
