Work in progress Automatic Lip Sync

This is a quick test of the first hackish pass at doing automatic lip sync based on phoneme recognition from PocketSphinx.

I had reluctantly set the idea aside due to poor results reported by UE4 forum user ShaneC, who wrote a nice wrapper for PocketSphinx as a plugin. After some fiddling around and much searching of usergroups about Sphinx, I was able to determine that passing in a blank option for the phoneme recognition file makes it work pretty reliably for some reason.
Playback of the visemes is a bit choppy right now for a number of reasons, mostly to do with just because it hasn’t had a pass to make it look nice, and because I’m not passing through the information about how long each phoneme lasts.

Needless to say this is completely pre-alpha and doesn’t represent the final result, just an extremely early first test.

Left 4 Dead 2 Style Wound System Complete (Mostly)

Great news!

The Left 4 Dead 2 style wound and dismemberment system mentioned previously, which will be powering the gruesome world of Dungeon Survival, has now been completed (save for a couple of odds and ends, and art assets).

Previous screenshots were taken using a hacky method of transforming the hit location into mesh-relative coordinates, whereas the proper way of doing it, and really the only way that truly works, is to use the “pre-skin” vertex positions.

For non-developers this essentially means getting at the points that make up the character mesh before they’re transformed by the animation rig.

The effect of this is that the opacity-mask sphere which creates the “hole” in the character mesh so that the wound geometry shows through will follow along with any animation properly.

This meant diving into the guts of Unreal Engine 4’s material system, right down into the proto-shader files that underlie familiar visual node-based materials. Luckily this ended up being quite painless!

Remaining work to be done on the wound system itself is to use a blood mask to dirty-up the edges of the character mesh where it’s hidden by the wound-ellipse.

Other than that, all that remains is to create wound meshes for the characters and hook up and tune the system.

As an aside, I decided to work on this today because work on motion controller IK has stopped for the moment while the new full-body IK with limits plugin is debugged a bit by its author for my use case.

I can’t wait to show off some of the gameplay with the new full body IK and motion controls, as it’s something you don’t often see outside of triple-A games that use software like Havok Behavior or Autodesk HumanIK, and is perfectly suited to VR avatareneering..

Grass Bending Test

Note: The following video is pre-alpha test footage, and in no way represents the quality of the final game.

It’s hard to see due to the video framerate and compression, but there are volumes for affecting the grass parented to the player’s feet.

Credit for the effect goes to Daniel Wenograd.

To add this to your own Unreal Engine 4 project, check out Daniel Wenograd’s post on the UE4 forums.

Storyteller – Fireside Tales featured in Vice article about Tripping in the Rift

Connington’s first VR trip was closer to what you’d typically do after taking shrooms. After donning the Rift headset, he was sitting in a cave around a crackling campfire listening to an old man reading George Martin’s A Song of Ice and Fire (the series that became Game of Thrones). The simulation, Storyteller – Fireside Tales, is an immersive audio book that makes it feel like the story’s being told by someone sitting next to you.

“The echoing of the old man’s voice through the cave, the dripping of the water behind me, and the warmth of the fire in front were so intense and real that I felt like I could reach out and touch them,” Connington says.

“I had left my student life behind and become part of the ASOIAF world, feeling like Hodor could walk in at any moment.” he says. “I started to dream I was Bran, stuck in a cave somewhere North of the Wall.”

Screen Shot 2015-11-18 at 11.46.09 AM.png

Image: Oculus Rift simulation Storyteller – Fireside Tales

Read more: http://motherboard.vice.com/read/real-drugs-virtual-reality-meet-the-psychonauts-tripping-in-the-rift

Thanks to Motherboard/Vice and Jon Connington for the shoutout!

Quick and Dirty Razer Hydra Support in UE4 Blueprint

Edit: To use the calibration feature, stand in a t-pose (arms at shoulder level extended to either side) and press the calibration button (start by default).

It doesn’t matter what pose your character is in during calibration as long as they are generally standing.

Many thanks to Getnamo for his Razer Hydra (info and download) plugin and excellent help figuring out how to set this up (more detail about calculations). I was definitely overcomplicating things.

Copy the Hydra Plugin into your project dir and enable it in the Plugins menus, restart.

Make a new BP derived from HydraPlayerController. I called mine DHydraPlayerController, D being for Dungeon (Survival).

Go to World Settings and set the game mode override settings to use your HydraPlayerController BP you just made.

GameMode Override settings for Hydra player controller.

GameMode Override settings for Hydra player controller.

Event graph for Hydra player controller.

This is in the Hydra player controller. Notice the variables there and the function Calibrate Hydra (I use start, easy to find on the controller). The base offset one is just a static 40 in Z, it doesn’t change. I could use a float but I plan on doing a more thorough calibration step in the future to get more exact mappings, so I’m leaving it as a vector for now. The midpoint we calculate.

Event graph for Hydra player controller.

Event graph for Hydra player controller.

Next comes the Calibrate function, also in the controller:

Calibrate function in Hydra player controller.

Calibrate function in Hydra player controller.

Next we’re going into the character’s animation blueprint

Open your character's animation Blueprint.

Open your character’s animation Blueprint.

First, add these variables

Character anim Blueprint variables.

Character anim Blueprint variables.

Then create this (looks messy but is simple). Pull off the character node to get the controller.

Set positions in the animation Blueprint event graph. Pull from the character node to get the Controller.

Set positions in the animation Blueprint event graph. Pull from the character node to get the Controller.

The last step is to set up something like this in the AnimGraph. Ignore the wiring that goes off screen, it’s just doing a static rotation to fix the default hands for my particular character.

Set up the hand IK using the animation blueprint variables from before.

Set up the hand IK using the animation blueprint variables from before.

I’m using component space for the IK target positions.

I’ll be porting this in a more polished state into the VR Game Templates by Mitch (of Team Metatron).

Porting up to Unreal Engine 4.3 – Setbacks Caused by Boneheadedness

Click image for source! (Credit goes to the artist! Not affiliated with the project - used for blog enhancement purposes)

Facepalm – Whiterun Guard

 

Progress has been swift but not without setbacks, as with the problems I encountered moving my project from UE4 4.2.1 to 4.3 preview.

I decided to port up to it because it offers some particular functionality I required, namely the On Actor Fracture event.

I use this to figure out when the destructible oil barrel has cracked open, to spawn the oil spill (which as described previously (under Other Stuff That’s Done) – can be lit by throwing a torch onto it, or other on-fire item). There are some other improvements I wanted access to, but that was the reason for switching now.

Unfortunately I made a bonehead move and converted a copy of the project, started working immediately on adding new stuff (torch throwing, barrel breaking spawn oil spill etc.) without actually checking to make sure that everything worked properly otherwise.

It did not.

The plan for last night had been to take a short video of how the gameplay with the oil barrels and throwing torches was shaping up, but instead…

I spent last night debugging the things that were wrong I had missed, mostly related to the item pickup/drop functionality over the network. My plan for today is to finish up going through the 4.2.1 source blueprints and make sure everything is intact and correct.

What seems to have happened is that the conversion didn’t actually get all the blueprint nodes and variables, and in some cases changed the options of other nodes (raycast was set up to show debug, despite debug being off in the 4.2.1 project).

Once everything is back to normal in the 4.3 project, I’ll be working on overhauling the inventory system.

Here’s the Trello task for that:

Inventory Overhaul Task on Trello

Inventory Overhaul Task on Trello

Once that’s all finished, it’s on to setting up the rat enemy and then the AI for it.

Rats! And You, In Dungeon Survival

Rat Swarm by Jay Odjick

Rat Swarm by Jay Odjick – (Not affiliated with the game – just for looks! All credit goes to the artist!)

Rats are usually boring enemies in fantasy and medieval games, used to introduce new players to combat, and hardly intended to inspire terror. The exception is usually some sort of giant rat, but that too has limited scare factor when the players are used to fighting giant-everything (spiders, wasps, bats, etc.) any of which is more creepy than a giant rat.

In Dungeon Survival, I want rats to be the first enemy players encounter, not because they will be pushovers, but because they will have some interesting mechanics that will heighten their fear factor and hopefully, do justice to how creepy a bunch of rats in a dungeon would be.

In this post I’ll be going over two aspects of rats in the game, though the first will be about the player.

The Kick Attack

Your character will feature a few basic unarmed attacks. I’ll be going over the kick attack now.

* Physics based – It will work by having a physics volume follow the animation of the kick on the character.
* When the volume connects, it will impart damage on the hit actor, as well as an impulse. Any object that responds to physics will fly out.
* Enemies that take enough of an impact on this stage will go into ragdoll mode.

Rat Behavior

One thing that almost anyone instinctively fears is disease, and in the case of rats, that means them crawling onto you.

* Rats will be able to crawl up the character.
* Using hand motion controls (or a contextual punch attack – more on that in another post) the player can slap the rats off, but it has to be done with enough (in game) force to actually move them.
* Similar to the kick attack, a strong enough hit will make the rats ragdoll.

In addition to that, rats and other enemies will take impact damage – so a good kick off a ledge can be a one hit kill if done properly.

(Those readers interested in VR will note that the rat climbing behavior outlined above will be particularly creepy in VR. This is by design.)

Might look something like this.

Swarm of Shadow Rats by Christopher Burdett (again - not affiliated with the project, this is just to illustrate what I mean!)

Swarm of Shadow Rats by Christopher Burdett (again – not affiliated with the project, this is just to illustrate what I mean!)

Plans for Early Access, Modding and the Future

In this post I’m going to cover some of the longer term plans for development of the game and also how I’m going to be making it available for players.

First about the demo/early access situation.

In my own game-playing experience, I find that there are quite a few indie games that are either released or in an early access state that are great but don’t offer enough real meat to the experience. To be expected from alpha/early access games, but it is a lamentable situation.

Especially with co-op games, it’s hard to get friends on board to ultimately run out of content and things to do.

One game that I think did a great job handling early access was TaleWorlds’ Mount & Blade (a game I am particularly fond of).

The initial beta release had enough gameplay to keep folks interested and to grow a vibrant community.

However, M&B didn’t really explode past the initial word-of-mouth popularity until that same passionate community figured out how to mod the game by reverse engineering how various aspects of it worked.

A frenzy of activity was spurred by the first modders, and eventually tools to mod the game more heavily were developed by the community. Finally, some months later TaleWorlds released a more feature complete way of modding the game via the module kit.

To this day M&B has a strong modding community and an active player base.

Moving off of the segue…. What does this mean for Dungeon Survival?

As with many indies, the plan for Dungeon Survival is to ultimately launch a Kickstarter to fund the game’s development out through release. Unfortunately though, that doesn’t jive with my desire to get the game into the hands of players as soon as its fun.

Right now development is focused on building out a 3-level dungeon demo which will show off gameplay in a specific way:

* Basic survival gameplay – eating, drinking, using the environment to kill or avoid enemies. This will be on the first level of the dungeon, primarily (and through the 2nd and 3rd, of course, but the 1st level will be focused on those aspects).

* More advanced inventory and environmental interaction, as well as crafting.

* Finally, the third level will focus most heavily on challenges and puzzles that require the player to use the environment to overcome them, as well as a finale boss fight to get to the exit from the third dungeon level to the fourth (which will be the end of the demo).

This demo is what will be shown as part of the Kickstarter campaign, as well as for promotion in general (screenshots, trailer etc.). The game’s art and assets will not be final, and gameplay will not be totally complete. But (hopefully) it will demonstrate what the game is about and be fun.

To that end, my plan is to get:

* Enemies and more environmental gameplay in the game.
* The first dungeon level.
* Some traps.
* More items and gameplay related to items.
* A basic save/load system.

Into the game as rapidly as possible (check out my previous post for info on the current status of the game’s features).

Once those aspects are implemented, I will be getting the first alpha of the demo out to early beta testers for feedback and bug reports.

After a thorough round of testing, the demo will be released for public beta testing. Iteration will continue on that demo, fixing bugs, adding gameplay, and building out the second and third levels of the game.

Once all three levels are in and it’s fun enough to replay for a couple of hours at a time, I’ll be starting up the Kickstarter campaign, and releasing a public demo of all three dungeon levels.

Shorter term I’ll be setting up a page on Indie DB, as well as otherwise promoting the development progress in places that makes sense (Reddit, Unreal forums etc.).

Soon to come in the next few days are video of the current gameplay and some of the first little bit of dungeon level, a sneak-peek of the dungeon level layouts and more.

A Word About Modding

I strongly believe that mods can make or break a game’s success long term, and to that end I plan on supporting both a ton of customization of the game by default, as well as mod support.

Unfortunately, this presents some interesting challenges. Namely, Unreal Engine 4.

As you can imagine, the best way to mod an Unreal Engine 4 game would be to have the engine itself, and the .uassets that make up the game. Unfortunately, there are some legality and practical issues that come into the equation because of this.

For one, as part of the Unreal Engine 4 license agreement, I am unable to ship UE4 editor tools along with my game for modders to use. Nothing unusual about that – after all, they don’t want people using mod tools from a UE4 game to make their own stuff without buying the engine.

Secondly, distributing the game’s assets. There are some issues with this, which I’m not really qualified to speak to. Suffice it to say that there are some restrictions on what can be done.

How do I plan to work around this and support mods for my game?

To start, whereever possible I am going to make the game use plain old data for configuration of items and things like that. So modders will absolutely be able to go in and change values/add new items that utilize the existing gameplay functionality that items will provide.

Most modders should be able to use that to get what they want. It’s when that isn’t enough that things become tricky.

So for those brave modding souls that want to add new interactions and gameplay that are not in the game and supported by the item system already, you’ll need to do essentially 2 things:

* Get an Unreal Engine 4 license – it’s $20, you can get it once and cancel the subscription immediately.

* Download the stuff that makes up Dungeon Survival, or at least, pieces of that stuff.

I don’t think this is too bad, but it’s also not totally ideal. I’d prefer modding tools to be free if possible. Right now I don’t think it’s possible.

However, as mentioned I will support as much modding out of the box as possible, both through in-game options as well as tweaking/adding data through the game files.

The Dungeon Of Death! (and gameplay)

In Dungeon Survival, we must provide a counterpoint at all times to the mundane aspects of survival, maintaining one’s health in an environment of slimy subterranean walls, and avoiding the denizens of such places (or dispatching them with one’s ingenuity).

I am speaking of course of the all encompassing dread reaper: Death!

In other words, the Dungeon Survival project now features death and respawn.

In addition to this, since the last update (which was somewhat out of date) the game has:

Inventory System

This is sort of the basis for much of the game, as most of the objects will be “smart” – with some gameplay associated with how they work or interact with the environment/other objects in an intuitive way. I’ll follow up with a good example of this in a minute.

Items when on the ground are physics objects, much like in the Elder Scrolls series. When picked up, the object is attached to the player’s hand.

Currently, only one item can be carried at a time, not counting the “utility” slot (the right hand – which is at the moment occupied by the torch and isn’t completely hooked into the inventory system). In future, the player will be able to craft pouches which attach to the player and will be able to carry certain amounts of items of their own, depending on size/type.

As with the rest of the game, all of these things are networked for multiplayer – including kicking around physics objects (this may change depending on if it presents problems later) and having it sync to everyone.

When picked up, the item is not destroyed – it is taken into the character’s inventory, and attached directly to the hand (or will be “stored” in pouches if available). This is important because the item system in the game will also be persistent – a half-eaten apple will stay half-eaten even when dropped (and as that example implies, the player does not eat food all-at-once). Items will have persistent state.

The above behavior is for the default item type. All specialized item types derive from this type of item and include its behavior (though in special cases they may override it).

As of yet there is only one specialized type – the food item. Which brings us to…

Hunger and Thirst

First, some rationality for even including what some might consider a micro-management gameplay feature.

I’ll always remember my first few forays into the realm of Nethack, that character-based dungeon crawler with timeless charm and a rich set of gameplay.

In particular was when I first made it to level 3 (this was probably not Nethack itself but a derivative, so forgive inconsistencies, those Nethack aficionados in the audience). I had no food. I was dying of hunger, and a dwarf attacked me.

I dispatched the little fellow with little health left to spare, crawled around a corner and took the only option that remained to me. I ate the dwarf.

I ate his corpse and promptly died from dwarf-poisoning.

For those that don’t know – Nethack and its like feature “permadeath”, meaning you must start over if you die. Instead of being (justifiably, since the 3rd level of the dungeon was the farthest I’d ever progressed) irritated, I was cracking up with laughter.

I loved the fact that I was able to improvise in that way and eat the body of a dwarf I’d slain. And even more that it distinguished poison from normal food.

That’s why I wanted to make the Dungeon Survival project in the first place – to capture the magic of the “emergent gameplay” that roguelikes provide, but with a focus on natural interaction with the environment, and on survival gameplay (with, of course, VR heavily supported).

What’s done:

* Hunger ticks town every 5 seconds (numbers here are subject to change with testing).
* When hunger runs out, the starvation meter kicks in. It ticks down ever 5s as well, but at a different rate (you have more starvation meter than hunger meter, but it depletes in larger chunks).
* When your starvation meter runs out, you start to take damage to your health.
* As mentioned before, food items exist and can replenish your food meter.

Thirst and dehyration work in a similar way, but are not hooked up at the moment to aid in testing (and because I haven’t hooked up any drink items, though that doesn’t take much time).

To come:

* Cooking, and uncooked food causing sickness.
* Starvation and eating too quickly after having starved for a while will also cause sickness.
* Poisoned/contaminated food.
* Dehydration and drinking too much water when dehydrated causing sickness.

Other Stuff That Works Now

In addition to all that, I’ve added:

* A proper blend for the torch hold animation, so that you can right click to sheathe/unsheathe the torch. The character’s arm will smoothly blend up and down.

* An oil slick blueprint, which includes an oil decal, trigger volume and fire effects.
* Code so that if the player walks over an oil slick with his torch out, it will light on fire (and does damage).
* A destructible barrel – damage accumulates over time, so that if thrown/hit against stuff enough it will shatter.
* An oil barrel version that spawns an oil slick when it breaks.
* A torch physics item using the inventory system, and the ability to throw the torch via clicking without anything in held in the left hand inventory slot.

This is the sort of “smart object” gameplay I referred to before. Eventually things like the barrel spawning an oil slick, the slick being flammable, and items that have fire lighting the slick up will be generalized so that items will simply be able to specify what types of smart gameplay they use (by both deriving from special item types and settings in those types).

The basis for this system is already in place, and is how the food items work – but currently the torch and barrel do not use the system (that will change).

To come in the short term:

* Thrown torches will light oil slicks on fire.
* Throwing your torch will remove it from your hand,
* Being able to pick up the torch to re-equip it to the utility/right hand slot.
* A two-hand pickup system for picking up things like the barrel (luckily though I’m no animator, I’ve figured out how to build an animation for that and make it work myself).
* Being able to throw two-hand pickup items.
* Kicking/punching physics objects, like the barrel.

After that, I’ll be adding the first enemies to the game (rats!) and building out the first level of the demo dungeon.

Soon I’ll be taking some screenshots and video of the oil barrel gameplay.