Analysing Cinematography

Who ever has played Silent Hill knows it has quite the impressive camera work to say the least. It's quite challenging replicating this as an amateur / hobby programmer. They use varying styles of cameras from dolly to pedestal to even: truck, pan, tilt and even zoom and more!
So to fully understand and re-create all these I've done a lot of play testing and broke down various stages to see what is used and how.

For example, Alley B (Hell) is quite complex in that it 8+ different cameras throughout this small section. All of which work both directions.

Above you can I've broke it down into sections:

1. Camera 1:
   This is a simple static camera upon entering the stage, it doesn't move but does zoom in as part of a small animated scene. The first change occurs at the intersection between this cam and Camera 2 A.
2. Camera 2:
1. A:
   This camera starts further away from the player, as they move towards the camera it uses a pedestal move to raise over the players head depending on distance.
2. B:
   Continuing from A when the camera is at its highest it then merges to B, which then follows the player as they move along the X axis.
3. C:
   Upon reaching the boundary between B & C the same camera is then forced to a single point yet is given more freedom to look at the player as they move around this small squared area.
4. D:
   Camera 2 then unlocks itself from point C and follows the player along the Z axis until reaching point E.
5. E:
   Here the camera frozen in place again but is allowed to look at the player as they move towards Camera 3 A.
3. Camera 3:
1. A:
   This new camera follows the player along the Z axis akin to Cam 2 D.
2. B:
   When the player reaches B, a cutscene is activated, after which A is resumed.
   Afterwhich the cameras can now be re-instated in reverse order and even back again.

All of that camera work is just for this one small scene. The heavy lifting is done by one singluar entity: Camera 2. This camera is altered throughout via triggers which fire specific code I made to mimic certain camera styles.

These two code snippets merge the current cameras position and rotation to a specific point and the other allows it to move along with the player respectively.

All these triggers are used for the camera movements detailed above!

This is just one section of the game, the others all use different styles throughout and the same process is used for them. Although this particular area is the hardest.

This video shows all this work put into effect. All assets have been removed so that focus is primarily on camera work!

Also as an added bonus, the video also shows off audio blending. The background music shifts intensity at specific points to cause anxiety the further in the player gets. This area has 4 different music triggers at points: Cam 1, Cam 2 D, Cam 2 E and Cam 3 A just before the cutscene!

Enemies

After a lot of frustration and my complete detest of animation, I've managed to make some progress on in essence the only enemy required.

Because I have the models from the original with animations you would assume this process would be straightforwrd. It is not.
Every character model is created using an old method. Originally to make body parts blend better models / meshes (body parts) were individually attached to bones, which at the time works well enough. A good example of this is Mario from the Nintendo 64:

All those seems on Mario are individual body parts welded to specific bones and animated that way. So in relation to Silent Hill, these models follow suit. So when importing one of them, this happens:

This makes it really hard to translate this collection of body parts into a newer rigged model. Because some models have baked in animations I can find a frame which best represents a idle pose and go from there.

Here you can also see different body parts in different colours. This particular frame is the closest to a standard pose I can find. So from here I combine the model, but doing so removes any animations sadly.

Now I can edit it as required and re-rig the model:

After all this, I placed it ingame and give it the same animation set as the player and here is the outcome:

So while its a big step, work still needs doing. I need to try and recreate some of its animations for attacking yet. But as is, I can work with this.

Polishing-up Little Bits

Currently 3/4 areas are now done! Some are missing little bits here and there but overall, these areas are playable. Now I'm adding 'polish' to the two Alley areas to really make them feel authentic. This entails more texture work like details, misc models to make other assets look genuine and cinematics!

Progress Tracking:

• Café
• Models / Textures: 90%
  Need to finish 3 models (TV, Coffee Machine and a Wooden Cabinet)
• Cameras & Triggers: DONE
• Interactions: Done
• Cutscene: 30%
  Exit cutscene mostly done, still debating doing the full conversation cutscene upon entering the scene.
• Alley A
• Models / Textures: DONE
• Cameras & Triggers: DONE
• Interactions: DONE
• Cutscene: N/A
• Alley B
• Models / Textures: 87%
  A wall, some fences, wheelchair and the floor need texture rework
• Cameras & Triggers: 95%
  Some cames need tweaking very slightly.
• Interactions: 60%
  Everything done bar a simple enemy(s) at the end to cause a fake death/
• Cutscene: 80%
  Main cutscene is done bar tweaks!

Here is a video of the cutscene from Alley B:

 Here is a link to a Youtubers playthrough of the same cutscene for comparison: Silent Hill Walkthrough the point of interest is at 5:45.

Re-modelling a Gurney

Heres a quick video/gif I made of re-creating the cloth covering a dead body on a gurney. I recorded it because I love how easy it is and how good the end result is!

The texture was very simple too, just two layers of a dirty paper texture overlapping with a linear and some blood splatters.

Progress Update:

Here are a few screenshots of the overall progress of the project so far. Currently I'm mid way through the third out of four areas. Once this is done then all small areas of the game are complete, leaving just the larger open area of Silent Hill itself.

In regards to the written aspect of the semester, I have all four of my references; ranging from professional(s) from Ubisoft / Cloud Imperium Games to Splash Damage and even a developer linked to my previous Rush 2049 project and finally an enthusiatic modder related to my projects / thesis.

Re-Designing a Dated Control Scheme

Throughout this project, I've had to make design choices that I believe should be eloborated on in more detail. Its one thing remastering a game, and a totally different one to remake it.

Silent Hill uses a control scheme dubbed "tank controls" which at the time was very common for this genre of game and many others followed suit like: Resident Evil, Dino Crisis, Parasite Eve etc. Even other massively popular games not in the horror genre used them too, like Tomb Raider for example!
This is ultimately down to hardware limitations as analog sticks wern't fully introduced during this generation of gaming.
Because of this, the design of most games had to follow basic movement input and designers had to come up with creative ways to use the most out of this in more advanced open worlds.

Even through next iterations of hardware, some of these franchises still used the same control style, simply because players had grownup accustomed to this input and it worked well for this genre.

In the industry today, this control scheme has ultimately been replaced with other methods, since control is much easier to implement in a variatey of ways. Tank controls are still in use, but mostly in niche titles, aimed purpsefully at recreating this type on control.
Other franchises in the horror genre have migrated to other control schemes such as the very popular "over the shoulder", some have even taken this further by introducing virtual reality as a method of control.

Silent Hill in particular in the later releases have changed to a more standard 360 movement model, allowing direct movement in any direction at the push of a button / stick. The latest in the series was going to use VR as mentioned above but was ultimetly cancelled.

So that leaves this project. What direction should the control scheme go? Should it be left alone to wholefully recapture the uniqueness of the original game? Or, should it lean more towards the other titles later in the franchise and allow a more modern system? 

Ideally both! However, due to time this won't happen. I would personally have had the option for both input types. Instead I decided to tackle this problem a different way...

Seeing how this is a remake, players new and old would both expect everything to be modernised; visuals, gameplay, controls etc. Younger players whom have never experienced older generation of consoles would immediately expect the game to control like any other modern title using 360 degree movement. Even older players to some degree would expect this new type of movement seeing how they would be accustomed to playing in this style for generations now.
This then brings to question the authenticity of the project, in regards to nostalgia, people who have played the original will expect to have the tradational controls too.

Balancing these arguements is tricky.

After a lot of testing and being a fan of both control schemes I believe I've designed a good compromise of the two.
Immediately I would rather the control scheme be similar to what the majority are use to: 360 degree movement. Now, Silent Hill encapsulates two styles of the tank control movement, the first being static controls for fixed cameras and the other 3D for the more open areas of the town.

With this in mind I've made it so that the player can move in any direction and the camera follow / orbit as required, with additional control to orbit the camera manually. This is what a modern title would play like, additionally I've made the camera 'chase' the player like it did in the original. So when the player moves forward, the camera is pulled, but if they move left or right the camera is stationary until the player moves in a direction again. On top of this, the original had a look-offset, meaning that the camera is always looking a foot infront of the player direction, I've also included this.
So the open area camera is modernised, yet still has behaviours reminiscent of the original.

The other movement style with fixed cameras is also a bit different. Originally because the player moves in a set world-direction, they could run through camera triggers seemlessly and get through sections fast.
To replicate this, I've still included the newer 360 movement, so the player can move relative to any camera, allowing easy navigation. Although going through a camera trigger would cause a switch in controls since "relative" controls are just that, relative to each individual camera. So what is up in one scene might not be up in another.
A way around this, whilst still mimicing tank controls is to delay any realtive control update. This means that the player can run through any number of triggers and the control will be the same, meaning they can keep on going without interuption. If and when the player no long wishes to pursue their current trajectory and the player stops, then any new movement is now relative to the current camera and the process can begin again.

I think this is a nice way to modernise the traditional tank control scheme. While it may sound complex and is hard to describe, playing it feels right.

So in the end I beleive I've solved this issue, I ideally would have liked an option for both inputs, but since this is a limited vertical slice it doesn't seem warranted.

Texturing & Workflow

At this stage in development I'm mostly re-creating assets again now. It is a bit tedious, especially trying to create similar visuals. A lot of the buildings I'm working on don't require improving model-wise, most are just huge blocks.

So I'm putting more effort into texturing, but again, a lot of detail is missed due to the fog, which I guess is both a blessing in disguise and a hinderance.

At any rate, the original models throughout the entire game were created on a grid and textured the same way. By my estimation they used a 32x32 grid. By creating models this way, they can severally condense textures into a single sheet, this allows them to reduce soo much waste and also to tile anything easily.

Here is a sample texture from the game:

The full size is 256x256, divide this 5 times and you end up with individual blocks (32x32). If a model deosn't use the whole sheet, other assets can have part or even share the whole sheet. This is a fantastic way to optimise assets and even their workflow.

The end result of the texture above looks like this:

Now because I'm improving quality throughout I can't just upscale these textures, well I could but tiling would become very noticable and I wouldn't be able to achieve the same level of quality I'm aiming for.

Instead, I'm taking individual models, optimising them, adding appropriate mesh details, like: window frames, side panels, vents etc (all which would been included originally in textures instead). This allows me to give models their own individual texture sheets of anywhere up from 1024x1024, which is 4 times the original quality, with some models having a 2k texture.

This is a UVW map I made from the model above, as you can see from the 256x256 texture, it's much different. There is quite a bit of black space on this particular one, but thats due to the size of the building and keeping each face to scale.

So with this, I then go on and find textures I think would suit and then begin layering and adding affects.

Using this texture as a base, I can now start transforming it to match my UVW map. I chose this particular one because the original model looks like it is made from large sheets of metal.

This is the same texure from above, just cropped, scaled and with some shading. Next I need to re-create the grungness of the texture:

And then add a nice opacity layer the break up the base layer, so it looks more natural:

Then add some more base layers for the other parts, like the door and steps, and add more gungle effects and shading:

Here is the process of textuing from start to finish:

The final model with this texture now looks like this:

It looks very close to the original I think, even a bit more realistic.

So basically that is it, this is my workflow that I use and will continue to use on the next 100+ models! Though I am aiming to only do specific models I know will be seen. Other models I can and will use the original PS1 ones, simply due to the fact detail are lost in the fog, so buildings in the distance are usually only silhouettes.

Objects the red arrows are pointing to are PS1 unedited models. The floor, garage, trash bins and building to the far right are all re-created.
Because of the area, these buildings will never been seen fully thanks to the static cameras. So I'm effectively optimising by not recreating them (they themselves optimised to removed extra verts / faces). Plus these models don't and will not have use for normal maps etc, so this is saving on memory too!

So, like I said before; some objects do benefit from the facelift and lend themselves to create a better quality game overall.

Technical Obstacles

For a couple of weeks I've tried to get around an annoying issue I've had with textures in-game. Basically, to achieve greater levels of detail I've been using multiple materials per object. The first usually a basic ground texture like gravel, then on top specific details overlayed i.e. a street with road markings.

This can be manually textured all into one texture sheet, but then the quality can be severally compromised, especially over large areas.

As you can see, even though the gravel is a 4k texture, because the area is so large it quickly becomes muddy. However, to get around this I can simply put the same texture into the material without first processing it into a traditional texture sheet. By doing this I can tile the texture across the whole area with no loss of quality:

This is the same area, just with separated textures. Now because the gravel is on its on material and tiled I can place an overlay with the original uvw texture but remove the muddy gravel. Alas, this too isn't without issues:

The textures may look better but the transparent overlay layer with the road markings are now no longer read by the renderer, making them visible regardless of the fog. After some reading, this turns out to be an issue with how Unity handles standard lighting and that any transparency pass is done last, resulting in the above look.

So after a lot of fiddling and research I did manage to over come this issue:

So overall it looks much, much better, the floor has a high quality tiled gravel texture and the roads also have a transparent overlay, which now is considered by the renderer, achieving the exact look I wanted.

This was fixed by using one of Unity's obsolete shaders (no idea why its obsolete, it is obviously still useful) and manually setting the render depth for this particular material.

Typically the "Render Queue" is set too 3000, by lowering it, the engine now analyses this material before rendering any Z-Depth aka the fog.

More Camera Progress

Cameras are one of the last main functionality items I need to finalise before I continue with re-modelling. That and they've provided a nice break from 3DS Max.

Previously I was testing vertical cams. I have made progress on this front. I was using Cinemachine in my last test with their virtual cameras. The issue with these is that they cause aweful snapping between other camera triggers and the player can also get caught in trigger loops and end up spinning on the spot. This is why I've focused more on Unity's default camera system.

Still though, I couldn't get nice movement with the default cameras, but I could with Cinemachine, because of its dolly track system. I did make progress in my last video using a "Dolly Camera with Track" object, but because it has a vcam, its very hard to integrate it with my other mechanics and still has the issues mentioned.
I did however find a nice medium between the two. Theres another object called "Dolly Track with Cart", this is exactly the same track system just without any cameras. So the cart acts as an empty object in which you can animate and do anything with:

 So, the first thing I did was attach a normal camera to this cart and see if it worked and voilà!

The camera properly moved in any direction I set the track as. The image above shows the track in a vertical position. Because it has waypoints and a current position variable I can move this how I need via triggers etc.

The next step was to make a script (still WIP) to access the aspects I need at runtime. So here I have the player to track, the cart to animate, which track it's using and the camera to affect, followed by values to edit and that feedback information to testing.

Finally to get it to where I need it to be I had to add a distance check from the camera to the player, this value is then fed to the carts position on the track with a modifier to increase or decrease the effect.
Here is a brief clip in-editor showing the camera working exactly like it does in the original:

Here is a comparison link to a playthrough of the game (its set to start right at this same moment), where you can see the camera's blue / Z axis mimics the exact movement in this video: Silent Hill Playthrough (5:10).

Next step is to tweak these vertical tracks to closer match the results I want, then finish up the rest of the camera triggers for these alley areas.

Camera Testing

Here is a quick video testing vertical dolly cameras for the last area. After this is its a matter of blending cinemachine cameras to achieve to rest of the looks

Small Update

Nothing big to report on just yet, mostly due to external circumstances. I'm just experimenting with modelling and tweaking gameplay to make it feel smoother.
Hopefully will have something more substantial by end of the week!

I did fix camera transitions, when the player goes from one cut to another, the cameras new transforms affect the players relative movement. I've now added a delay based on when the player stops moving to then run this update.
This allows the player to keep hold of a direction through multiple camera cuts without the player changing direction (Similar to the originals tank controls, but updated), that is untill they let go, then any new direction is fed to the player.

Last camera hurdle to overcome is vertical dolly tracks based on player distance to the camera. Once this is done, then pretty much the remainder of the project is visual-related; modelling, vfx, tweaks etc.

(SFW) Re-populating Old Silent Hill

Following on from my previous post. I've begun the lengthy process of re-modelling this small (yet very large) area of Silent Hill. In total there is a minimum of 70 buildings to re-evaluate, this doesn't include any texture variants, for which there are a few. this also doesn't include various props like benches, street lights etc, but these won't require much effort.
Since this is the largest part of the project I'm aiming to do 2+ a day which will take me about a month, because of this I'm, keeping the re-modelling as basic as I can and not added much mesh detail. Focus instead will be on texturing, this is because during testing I've noticed a lot of detail is lost in the fog anyway.
So, even though the models and detail will be better, it won't be very noticeable unless under close inspection, which isn't the point of the game.

If I think I may be running out of time, I will simply focus on buildings and props along the golden path that I know the player will see. Anything else such as misc buildings far out of the way I will re-use the original PS1 model. Because again, in-game thanks to the fog details are not really noticable and with it being in a newer engine the original models -buildings at the very least- look great upscaled.

So far I'm about 6 re-models into my quota, here is an example of one of them:

The Agernon (above), is one of the first to be made. I've used realworld textures to mimic the style of the original, I may re-visit this texture to add darker lines between the bricks yet.

Here are another two:

These 2 garage variants have both been optimised like previous models and remapped / textured with a much higher reslution. Sometimes the originals textures are too muddy to fully see any detail so I've tried to replicate them as close as possible.

Decontructing Silent Hill

Possibly the most time-consuming aspect of this project is modelling. The bulk of the game is based in the main town of Silent Hill aka Old Silent Hill as there are a few other areas of SH I'm not including.
I managed to get an export of the town from an online community and have been using it as a placeholder for now. This scene is soo big that it makes Unity freeze fora few moments everytime it's selected in-editor. Which I can only assume has a performance impact when running.

More to the point... I've spend an afternoon dissecting this area for better optimisation and management. I now have everything seperated or grouped depending if its lots of the same object. this is so that I can re-model one to replace all (such as street lights etc), or unique objects like buildings, which I can easily target to re-model.

Originally this area has a couple of extra streets, but because the player will never access these in my vertical slice then it is completely pointless even having them in, especially since the fog will prevent them from ever being seen anyway.

Since this is such a large area, if I focus on it now and get it all re done then that is a huge portion of the project done.
The Cafe is complete and the only other area is the Alleyway when the player breifly goes to Hell. Gameplay and functionality is pretty much complete, with only camera angles needing to be flushed out later on.

Optimisation

While going through the process of re-creating everything I will have to optimise as I go. Because all of these assets were made decades ago the technology used then isn't the same now, so during exporting from the original source material much more geometry information has been either gained through the process or was allready there.

This example screenshot shows the difference between optimised and non-optimised. This particular model is the roads, the areas I've higlighted in red is the original geometry. The rest of it is what I've optimised myself.
As you can see the whole upper left portion ofthe view has clean lines and isn't cluttered with lots of tiny tiles. With this being a fairly large model, this does undoubtedly affect performance!

Atmospheric Cameras

Currently trying to re-create some of the more interesting camera styles in the game. I've managed to analysis it enough to narrow it down to two base types that are used most commonly.

Dolly Track:

Dolly Track Targetted:

There is also a vertical variant of this style but it's basically the same, other more unique angles can be chained together using CineMachine.

These two styles though are prominent throughout the game and require a more practical approach rather than using the cutscene feature.

Cutscene Consideration

I've been toying around with cutscenes in my project, getting acquainted to the Timeline feature and Cinemachine.

After some consideration I think it best to not add these in, at least not for the moment. This is from a practical standpoint and completely unrelated to my hypothesis question.

I've decided not to include them mainly because this level of detail is too much for a lone developer to achieve in the timescale given. Cutscenes like many other things are usually given to a separate department to fully flush them out and make them visually appealing. I do not have that amount of time to spend on this, while I would like to it will consume way too much time, time that can be better spent on functionality and other areas. If I manage to get ahead of schedule I will happily re-look into this as I consider them surplus content for the time being.

When considering these cutscenes I have to take into account what I want. My goal is to re-create this game to play exactly like the original but with much better visuals inline with a modern title. Taking this into consideration cutscenes ideally would have to follow suit, which would be no easy feat.
I broke down into sections exactly what was required for me to decide if it was feasible.

1. Eye / Mouth rigging and animating
2. Old animations re-created and edited
3. Sound files for conversations
4. Examining original game to get a sense of timeframes

The latter two points are easy enough, even point 2 is doable to a degree. The hardest part would be to animate the entire scene to include new motions to make it feel more realistic. Such as arm movements to articulate points and even eye contact and mouth movement for words.
This level of work would require more time and dedication.

There are 3 cutscenes in the scope I want to achieve, each varying in difficulty. One of them I can add in very easily as it's a simple camera movement and some text. The other two are the game intro where the protagonist chases his daughter and then the café where he meets the police officer.
The café scene is the hardest is because this would definitely require a greater level of animations since both characters can be seen conversing.

The only other way I could do this, would be to mimic the original cutscenes; include audio, basic animations and add them to a timeline. But would potentially look too out-of-place, this is something that would need a lot of debating.
Not only this but the same condition can affect many areas of this project. Where do I draw the line on quality and where do I deter from the source in order to appeal to a new demographic?

...I am however going to use some cutscenes for functionality. They come in handy for triggering events and / or activating objects etc.

So for now, actual re-created cutscenes are not being done.

Cutscenes

Naturally to follow on from the subtitle mechanic I decided to see how cutscenes would work / play out. I've never used Unity for advanced stuff like this, but it doesnt mean it isnt capable.
So far, I've managed to tweak my subtitle system slightly to allow for it to recieve signals via the Timeline feature, meaning I can setup scenes and uses markers to trigger specific pre-defined dialouge & audio!

The two markers at the top correspond to the inspector view: Cafe#1 and Cafe#2 respectively.

 So when the first marker is hit, it sends the signal "Cafe#1", which runs to code I made just for this cutscene and plays the associated audio file.

So alltogether, when marker 1 is hit, it fires the "Cafe#1" signal, which in turn runs the routine "CadeCut1" in the script above and also plays the audio file "CafeCut2#1". The same goes for marker 2 in the timeline, which in turns runs the next routine.

To better take advantage of this workflow, I may include a cutscene selection trigger in my player interaction mechanic for quicker usage in editor.

I don't know if this is a good way to do this but it seems to work for my purposes, next to add player animations.

Final Subtitle Mechanic

After quite a long time I've gotten my subtitle system to what I would consider a final state. meaning that converastions are now implemented and all other aspects of interactions also work!

(Best viewed in fullscreen)

There are a few little things I need to tweak and some minor bugs, but all in all it is done and ready to be fully utilised throughout the project.

The whole system is possibly one of the hardest things I've ever programmed and I'm proud of it. I'm particularly happy that the way I implemented the dynamic conversation creating:

Since migrating to TextMeshPro I was able to implement conversations better. It uses a queuing system, so I can offload as many lines of text as I want its way and then I just filter how these are shown.

The hardest part was offloading the new method I created for my interactable script. It now has a list / array of publicly editable text entries, this can be edited on runtime too.

When the player interacts with this interactable object it behaves just like it does for the other methods i.e. it checks the other object for any message to display on keypress.
It checks what type of interaction it is, if this is a conversation type, it then knows to pull this new array. With this array it now checks how big it is, loops through each element and adds each entry to the queue to display.

Next major thing to focus on is cameras and then it's a simple matter or recreating the other areas.