3D!

Recently I've started to learn how to use the Unity engine. It has good documentation and accessible tutorials, so the learning curve is not steep. It has a basic set of standard assets, importing new assets is easy, and C# is a pleasant language for scripting. I think the main feature over developing my own engine is that the editor functions as a level editor too, so it's easy to start producing game content.

I wanted to try my hand at 3d games, so I created a simple demo game to see how far I can get, and decided to document it. It is one level of a puzzle game, inspired by The Talos Principle (which is an awesome game btw).

Sketch of the beginning area with actual models and some placeholders

I wrote my own scripts for FPS player control, mostly for learning experience, since the standard assets include such scripts. I also implemented the gameplay with scripts for controling doors and buttons, picking up (companion) cubes, and teleporting the player. Due to intellisense and the intuitive API, scripting is quite easy.

Added my first trees, grass, and tried to paint the terrain

There are some nice tools built into the editor that help creating basic content for certain kinds of games. The terrain editor can be used to sculpt and paint the terrain. Grass can be created as billboards from texture or from 3d models, and painted onto the terrain. The tree editor can be used to produce giant broccolis from mars, and also trees. It seems to use L-systems, and is an easy to use but versatile tool.

Fleshing out more parts of the level

I used Krita for drawing textures (it's a nice open source painting program) and Blender for modeling 3d assets. Learning blender is somewhat challenging. Due to the complex interface the learning curve is quite steep. I couldn't find very good tutorials either. The worst part is that most off the official tutorials are not free, which is pretty stupid IMO... how do they expect Blender to get popular if they restrict access to the learning material? Well, whatever.

Placeholders replaced with final models

It's not all sunshine though, I run into some difficulties. For example I spent quite a while trying to perfect picking up objects, and I had to settle with a mediocre solution (with object having physics disabled while picked up). I just couldn't find a combination of properties that didn't result in some physical weirdness.

The whole level from birds eye view

It's quite interesting to work on other aspects of game development (modeling, texturing, level design) than just programming. There are different kinds of problems popping up... how do I create interesting puzzles? How do I plan a realistic map? How to paint tileable textures? How to carve a cake out of a cylinder?

End of the level - it's not a lie

It's pretty cool for indies to have a full dev toolchain for free. Blender and Krita are open source, Unity is (mostly) free, and for coding MonoDevelop is open source, and Visual Studio is free. If only they all run on Linux...

Texturing my model in blender

I've also recorded a playthrough to have something better to show off:

It's not very spectacular, but I'm quite proud of it, since this is my first real 3d game. I always thought working in 3d would be too much work for one developer, but turns out it's manageable. Next I think I'll try to make an actual full game, it would be nice to have another full game developed besides The Silliness of the Lambs. I have many ideas, but must of them are too complex for a single developer. Also, my modeling and other art skills are not too great. Though at least I have time, and I'm having fun.

C++ string formatting, etc.

The lack of articles recently is because I became a professional game developer. Since January I work as a senior programmer at a major indie developer studio, and for a while I didn't have much energy to work on private projects besides work. The new job also meant moving to yet another country, and it took some time to settle down.

Although it's fun to code AI for bosses, improve the engine, gameplay, etc. as a fulltime job, after a while I felt like working on my own stuff, and recently started to experiment with an idea I had. I was wondering if a database would be suitable for storing all the data of a game, and to check it, I started to implement an engine with SQLite in it's core. I'm not sure how it will turn out in the end, so far I like it. It's convenient to have access to textures, sound files, configuration values, etc., or writing log messages with simple SQL queries, not having to handle files.

As a side effect I've also developed the core of a simple and easy to use string formatting solution which uses C++11 features. I searched for good solutions to support formatted logging, but didn't really like any of them, so I wrote my own string formatter classes, and decided to turn them into a full library and release it as open source, since it might be useful for others too. It's available at https://gitlab.com/almonsin/fmt11

I plan to continue extending the new engine and try to make an actual game with it.

Shaded Island

After writing the last article I have continued to perfect my island generator. I reimplemented the Gaussian blur to use separated kernels, which resulted in a huge speedup in the generation step, but rendering was still slow when the terrain resolution was big, so I wanted to improve on it.

I spent a long time on looking for the best rendering solution. First I checked out Ogre, which seemed to be a decent rendering engine. Unfortunately the configuration is very complicated and it's hard to find reliable, current documentation. I couldn't get anything get rendered on Linux with OpenGL after several hours of trying, so in the end I just gave up sadly.

Next I tried the bgfx rendering library, which also looked nice and much simpler than Ogre. This time my problem was that I could not get it to cooperate with SFML, and I did not intend to ditch that library.

The next choice was OGLplus, a C++ wrapper around OpenGL, looking good at first glance, but a bit complicated and bloated when I tried to use it. So after looking at options for weeks I kinda gave up. I considered using the vertex buffer and shader facilities provided by OpenGL, but was too lazy for it.

Then one day I accidentally discovered that the 2nd version of SFML, in contrast with the 1st version, has support for shaders. It's easy to load and use them, and the shader language is GLSL. It also supports vertex arrays, but unfortunately only with 2D coordinates for position, while I needed 3D. Luckily the SFML sources are easily readable, so I could figure out what's going on behind the scenes and learn how to use vertex arrays in OpenGL.

Switching to vertex arrays had a huge speed improvement, which is what I expected, since now there are no calls to glTexCoord2f and glVertex3f for every single vertex, only one call per frame to glVertexPointer, glNormalPointer, glTexCoordPointer and then glDrawArrays.

Using shaders enabled me to change from flat shading to Phong shading. It required calculating the vertex normals (which get interpolated in the shader for every pixel), not just the face normals for the triangles. The result is pretty nice:

Flat shading

Phong shading

Playing with shaders is fun, it opens up a lot of possibilities, for example I can modulate the colour of the terrain based on the height value (= z coordinate), e.g. set the lower parts to sandy colour and the higher parts to grassy green. It still needs tweaking, but the basics are done.

Another feature I have is the day night cycle. It was already present in the old renderer, but was very slow. For every frame the colour of every triangle was recalculated, based on the dot product between the sunlight direction and the triangle normal. In the new renderer the light direction is passed in to the shaders, and all the calculation is done on the GPU instead of the CPU.

Next I plan to implement vegetation generation to add trees, bushes, etc., and other features needed to make it into a game.

Journey to the Perfect Island

I took a two-day break from work to have a long weekend and some rest, and rest included working on a small project: island generation. My goals were simple, I wanted to make a procedural terrain generator that would create nice looking islands, and a triangle based isometric renderer in OpenGL.

First I calculated on paper the view transformation matrix to render a 2d heightmap as triangles, and wrote a wireframe renderer to check my calculations.

The view transformation was alright, so I added a noise texture and green colour to the triangles. First I also tried adding Gouraud shading, but it was ugly and the exact geometry couldn't be seen, so I switched to flat shading.

So rendering was sufficient, it was time to start generating the terrain. My plan was to generate a fractal heightmap, and then multiply the height values with a scaled 2d Gaussian function to have some land in the middle and quickly reach zero, that is, sea, as getting further away from the centre. After checking out a few methods (e.g. perlin noise, simplex noise, midpoint displacement) I chose the diamond-square method to generate the terrain, because it seemed to result in nice looking terrain quite fast.

It was not hard to implement it, and I was satisfied with the result. I also changed the viewing transformation from the skewed triangles to a more traditional isometric view. As the next step I have added the multiplication step with the 2d Gaussian bell curve, and also created the sea (and I saw that it was good) which is just a semi-transparent blueish plane slightly above the zero height level.

Looks good, right? Nope. It's the most boring island ever, a single pointy mount in the middle, an almost perfect circle shoreline, and most of the features from the fractal terrain are smoothed out.  I refined my expectations: interesting mountains, nice beaches and complex shoreline. As a fix I tried to add the Gaussian function to the noise heightmap instead of multiplying with it.

Now it's not terribly awful, just relatively bad. The features of the original fractal terrain are somewhat kept, but it is still just a single mount and a circular shore, no nice flat beach... far from perfect. I went back to reading articles and in the end found this one, which described a method similar to mine, but used another function instead of the Gaussian. It looked good, so I gave it a try.

Somewhat better, but still boring. I wanted something like this, with the constraint that it should be an island. The problem with that method was that it was based on Diffusion-limited aggregation which seems to be a looong process to me, and also the result is not always an island. After a good night of sleep I came up with a new and simple method: I would generate a random walk with Brownian motion to define the skeleton of the mountains, and apply Gaussian blur (you can't escape Gauss) a few times with different radii to smooth it into hills and shores. It took a while to code the Brownian noise generator and the Gaussian blur, but it was worth it. When I finally saw the first generated island I immediately knew that this was what I wanted.

This method seems to satisfy all my criteria: complex mountain structures, wide flat beaches, interesting shoreline. The shape of the island is mostly defined by the result of the Brownian noise, and the size is somewhat related to the steps used to generate the noise, though it can be stuck in a small place and result in a small island even with many steps. The implementation needs some tweaking (resample the terrain for higher resolution, rewrite the Gaussian blur to use a separated kernel instead of the O(N^2) slow naive 2d convolution, and fine tune the number of steps for the random walk and the radii of the blurs), but otherwise I'm very satisfied with the method. Now I can play with decorating the island, and soon I will have my own island with palm trees and beaches.

Concerning component-entity systems

Recently I have been discussing component based design with a friend, and he asked me how my implementation worked out in my platformer. My description grew quite long, so I thought I would make a post out of it - I wanted to do it anyway, I was just always procrastinating. And since there is not much practical info available about it, hopefully it might be useful for others as well. My system is based on this article, and probably this post will not be too clean without reading that one first.

In my C++ implementation Entity is just a typedef to int, they can be created by entities_create(); and recycled by entities_destroy(Entity i);. Currently there are 6 components: Common, Physics, Graphics, Animation, Audio, Camera.

Components

The most basic is the Common component, it contains the position, rotation and facing of an entity. I wanted to group together all the data into one component that most entities have and most other components access.

The Graphics component assigns images to entities. It accesses the Common component to know where to draw the image. The Animation component is quite similar, but instead of static images it uses animation instances.

The Audio component is also similar to the Graphics, it maps entities to Sound instances. At first I thought that it might be a limitation that one entity can have only one sound playing, but then I realized that I can create a new entity for each sound I want to play if needed. Simple.

The Physics part is more complicated, it consists of a main component which maps entities to Box2D bodies, and a subcomponent which maps entities to foot sensors (these are used to query whether an entity is standing on something). These are separate components, but due to the underlying Box2D implementation they are intertwined (the foot sensor needs to access the body), so I just put them together. A body can be either static (for platforms) or dynamic (for moving objects), so it could be possible to handle them as two separate components. Each turn the Physics component updates the position and rotation attributes in the Common component.

The Camera component is a bit special: only one entity can have it (that is, the Camera component has an Entity target; variable instead of a mapping from entities). I don’t remember the original article mentioning such components, but I thought it was logical to do it this way. Essentially it uses OpenGL transformations to follow The Entity so that it appears in the centre of the screen. Changing it to follow other entities can be used to simply implement nice effects, e.g. following a projectile and zooming in on it.

These are the components on the C++ side, but it’s just half of the story; most of the functions of the components are exported to the embedded Lua interpreter, so other components can be implemented in scripts. The C++ components are quite general and could be used to implement many kinds of games (at least I believe so), while the components related to the actual gameplay can be done in Lua. So far I have only implemented the Bullet component, but I was planning others (e.g. AI). Maybe one day I will continue working on this project.

Technical details

For each component I have header file to make all the public functions available to other components (and, more importantly, to Lua), for example common.h contains:

void common_init();
void common_add(Entity e, float x=0, float y=0, float angle=0);
void common_delete(Entity e);

And <name_of_component>.cpp contains the implementation. Private variables and functions are static, public functions are exported in the header, and public variables are global. Yes, global, the second most evil thing after goto, and just slightly better than premature optimization. Please don’t stone me… at least until I blow this whistle. Even if I do say "global". But to be serious, I think this was one of the biggest advantages of using a component-entity system: that it taught me how to tame my globals. I was always told not to use them, because who knows which part of the program modifies them at what time, so they can cause lot’s of headache. But with this system it’s quite clear which component uses what other components, so the modifications can be easily tracked (at least I can do it since I am the only developer… this might be different in a team). And anyway, passing around one huge object containing all the game state is no better.

To provide the mappings between entities and components I use two classes (T is the type of the component, usually a struct):

template <class T> class DenseComp {...};
template <class T> class SparseComp {...};

DenseComp uses std::vector to store the component, and it is used in the Common component. It works since a mapping from Entities (which are in fact integers) to component data is needed, and that’s what a vector does. Since usually most of the entities have the Common component, it’s not wasting space, and this way accessing the Common component of an entity (which is quite frequent) happens in constant time.

SparseComp uses std::map to map entities to a component, it is used in general for other components. It is more space-efficient than a vector when just a part of the entities have the component, and usually querying the component of a specific entity is not too frequent, so the slower lookup is not a problem.

Creating two separate classes for components might have been premature optimization, since currently I just have a handful of entities, so it does not really matter. Maybe using just the one with std::map would have been enough.

Using plain C functions and structs instead of C++ classes to implement components has a big advantage: it’s much easier to bind them to scripting languages. Most scripting language implementations support binding of C functions, but binding methods of C++ classes is complicated at best and usually requires some binding generator. And since Entities are just integers, it’s straightforward to use them in scripts, no need for wrapping classes or handling them as special user-data.

On a sidenote, I was experimenting with turning the engine inside-out, that is, throwing out the Lua interpreter, compiling the code as a shared library and then loading it from Ruby or Python or LuaJIT or any other language implementation which supports it, and this way making it available for other scripting languages. It was working fine on Linux, importing shared libs is a piece of cake with LibFFI in Ruby and ctypes in Python, but unfortunately I had issues on windows: loading the dll hung. The problem was related to using the SFML library, and I could not solve it even after a few days of debugging, so I gave up. I considered migrating to SDL, but it would have been much of a hassle.

Summary

All in all the component-entity system worked out really well in my engine. My codebase is well organized, related code is contained in small, well-separated units. Adding a new feature usually means creating a new .cpp and .h file and requires very little change in existing code. Extending functionality is also easy, since modifications usually affect just one component.

This design also hands itself well for building a base engine with general functionality and creating the specific gameplay in scripts. I have no experience with building a complex engine based on OOP, I can imagine that it would work just as well.

One thing that is not clear is when to group together simple components into more complex ones, or the opposite, when should it be advantageous to cut a complicated component into several smaller ones. I suppose it needs some experience to decide this.

I did not completely throw objects out the window: I still use them as basic building blocks, and they are great for that. But on higher levels there are just entities and components. I think it’s like playing with lego. When you build a house you use the small bricks, but don’t really design it on the level of bricks; you say that the house should have a kitchen, a bedroom, and a garage, and then design those with bricks, and decide how they could fit together to make up a house.

And remember, respect is everything

After almost a year finally a new article. I wasn't idling, but on the one hand I had lots of other things to do (changed job, moved to a new country, made many new friends and discovered new activities like wall climbing), on the other hand I could not make significant progress on any of my projects, so there was nothing to write about.

I did not abandon the platformer engine, now and then I was tinkering with it. I have implemented the audio component, so game entities can have sound effects and music can be played in the background; I have added support for joysticks; and migrated to SFML 2.1 from 1.6, which was not trivial due to api changes, but was not hard either.

After a while there was not much work to do on the engine, it was time to start creating artwork - I could not postpone it any longer. So I designed a character, painted the required body parts, scanned and processed them with Gimp, tried to make a simple walk animation, and failed miserably. It just couldn't get it look good. The main reason is that the animation system I designed did not turn out as good as I expected. Without a skeleton it was hard to keep body proportions and find the correct positions and orientations of the parts. So I designed another animation system based on a skeleton, but could not find motivation to actually implement it.

Slowly the decision reaped in me that I should change to a game which does not require much animation, and then I decided I would try my hand at a GTA style game. Not the modern 3d GTAs, but the oldschool GTA 1 and 2 with top-down view. I researched my possibilities for physics and graphics, and finally decided to modify the platformer engine to be suitable for this game style instead of implementing a new engine from scratch. I went into a three day coding spree, and in the end I had the car and pedestrian physics, pseudo 3d buildings and very basic gameplay implemented.

The physics is based on this nice Box2D car example which I reimplemented in C++. It uses a body for the car, and a number of steered and/or driven wheels. The wheels can be placed anywhere on the body, so many different vehicles are possible: cars with front, rear or all-four-driving, motorbikes with 2 or 3 wheels, trucks and limos with 6 wheels. So I did not have to reinvent the wheel.

The game logic and physics are 2d, but buildings are drawn in 3d, which gives a sense of depth; just like in the original games. Vehicles and buildings do not need animation, but I’ll have to implement it for humans. I plan an animation system based on separately drawn frames (like in The Silliness of the Lambs), nothing complicated (like my attempts for the platformer).

The gameplay should be full with action, gunfights, speeding, blood, gore, explosion, all that is fun. Maybe I’ll add RPGish features as well. But I don’t promise anything, I have many other things to do in The Outside, and I also want to experiment with some other ideas… so eventually something will happen to it.

 

Platformer - a bit more like a game

There were quite lot of improvements made in my engine, the most important is that I have implemented the animation system, working as described in a previous article. Most of the code was already written for the animation editor, so I 'just' had to port it to C++ from Java, and make the rendering work with OpenGL.

A small change that turned out to be quite important was that I have exposed some keyboard handling functions to Lua, so now it is possible to handle key events from scripts. It is important because it makes it possible (and easy) to add new gameplay features from scripts, without modifying the engine. The wizard should throw a fireball? Just add some lines to the keyboard event handling logic in the Lua script, which creates a new entity, sets up its physical properties (size, position, speed) and appearance (add an image to it), and done, there flies the fireball! As the engine becomes more and more complete, the more code will be written in Lua, and less in C++.

Now that the scripting part of the engine is capable of so much, I have added some mainly eye-candy features besides the projectiles: the camera follows the projectile and zooms on it; the camera target can be switched now, it follows either the wizard or the dog, but it could be any entity in the game; and the last new feature is shaking the camera after falling from great heights.

I have been using Bitbucket as an online backup for my projects for a while, and recently I have started to use it's issue tracker too, which is great. It helps me collect the tasks I plan to make in the engine, so when I decide to do some work, I just take a look at the issues, and I can easily decide what should be the next step. I don't have to remember all the possible improvements, and more importantly I don't forget my good ideas.

Another awesome thing is that I tried to compile the engine on Linux, and after collecting the necessary libraries and setting up the build environment, the code compiled without any warnings, and it runs fine on Linux too. I developed the whole engine with platform independence in mind, but I was glad anyway to see it compile and run without any problems on Linux too, especially since it depends on several libraries.

The engine will soon have enough features implemented to be sufficient for a simple game, so after some minor improvements my plan is to create some real artwork and a level, and make a short demo version of the game I have in my mind. Though it might be delayed a bit, because I am changing my job (even moving to another country), and I think it will keep me busy for a while.