The Audio Team at BBC R&D North Lab has recently published an orchestrated media demo (the Vostok-K incident) and is now available on BBC Taster (https://www.bbc.co.uk/taster/pilots/vostok). It shows how we can orchestrate media playback across multiple user devices to deliver a more immersive experience. The demo uses a cloud-based media synchronisation service that Dr. Rajiv Ramdhany (thank you for sharing the news, Rajiv!) built for the 2Immerse project. The demo is very similar to the one we shared a couple of years ago in a IEEE J-STSP journal article (open access). I believe a version of the underlying system (not sure if its the one used in the Taster demo) has borrowed our idea of a perceptual model to adjust playback for different “catch-up” scenarios (all complex equations can be found in the paper, if you are interested). Technically, it is quite difficult to achieve this level of synchronicity without using special chips and network protocols (read this paper to see how many things can go wrong). What’s also fantastic about this demo is that it uses content specifically created for the technology. Instead of using any off-the-shelf 5.1/7.1 movie sound tracks, the demo splits sound sources and merge them on-the-fly based on what user device(s) are available.
I think the biggest challenge is the level of human intervention still required for the demo to work and work well in the wild over mobile devices. Device discovery is an obvious topic and we can throw some crazy idea on it easily (e.g., ultrasonic piggy-back). I am also interested in how devices’ capabilities plays a key role in the experience (also observed from my previous experiments). While listening to the Vostok demo, I was subconsciously trying to work out my own location in the scene and that is often dictated by which device has a higher volume. So my personal experience might not be the same as how the directors wanted. Is this a bad thing? Not necessary. Like the user controlled 360 video, allowing the audience to choose where they want to be (e.g., decides which character you want to stand next to in a scene) can be a good pathway for content customisation and interactive media. Perhaps I should get my 3rd year undergrads to try out the demo in classroom. Wouldn’t it be cool to have 40 mobile phones going crazy at the same time? Maybe they’ll also come up with some nice projects to work on.
The vacancy can be accessed directly via https://jobs.northampton.ac.uk/vacancy/research-assistant-fixed-term-361582.html and will remain active until 11.59pm on 12 September 2018.
We wish to appoint a Research Associate to work on the EPSRC-funded project “Software Defined Cognitive Networking (SDCN)”. The SDCN project aims at enhancing online video distribution, which takes the vast majority of internet traffic. We seek to develop new context-aware network models to improve the user experience and network efficiency using software-defined networks.
We invite applications from enthusiastic individuals with experience in relevant aspects of communication networks. The ideal candidate will be a proficient programmer in languages appropriate for scientific computing (e.g. Python, Java), have knowledge of statistical modelling. The role will require you to engage with external partners and stakeholders, and therefore you should possess excellent communication skills with evidence of working effectively as an individual and within a team. You will have strong project management skills and have an ability to write up research for peer-reviewed conferences and journals.
You will work at the University of Northampton’s new Waterside Campus. A dedicated lab at the university’s Avenue Campus will also accommodate research and experimentation activities. The University promotes continuing professional development (CPD) and provides access to development for staff at all levels.
The candidate should possess a PhD in computer science or closely related field. This is a fixed term appointment of 12 months.
For informal enquiries, please contact Dr Mu Mu, email: email@example.com
Congratulations to Yoana (previously a BSc Business Computing student and now on our MSc programme), for having a paper accepted by IEEE VR 2018, a leading conference on 3D and virtual reality research!
Yoana’s research is centred around a simple but fundamental question: Does the “WOW!” effect of VR contribute much if anything to students’ learning outcomes in a higher education setup? While VR is increasingly adopted by primary/secondary schools in the UK to improve the pupils” engagement with learning materials as part of the STE(A)M initiatives, it is unclear how the technology would and could impact the learning of hard sciences in university. Yoana conducted a comparative study on students’ performance in a standardised assessment when course content is delivered using VR and conventional lecture slides. Interestingly, students see VR as a great platform to isolate them from real-world distractions but the extra cognitive load brought by VR content has a detrimental impact on how the learners recognising/memorising important quantitative data. We also concluded that social interaction and tailored productivity tools are two main factors that underpin the exploitation of VR in HE.
Slavova, Y. and Mu, M., A Comparative Study of the Learning Outcomes and Experience of VR in Education, to appear in Proceedings of the 25th IEEE Conference on Virtual Reality and 3D User Interfaces (IEEE VR 2018), Germany, 05/2018
Inspired by some interesting discussions with the game team at work, I recently picked up Scott Rogers’ Level Up! The Guide to Great Video Game Design in an attempt to learn how to design an engaging game. There are a lot of fascinating conventions, rule, and forms of thoughts in game design we can borrow in pedagogy, psychology, and immersive media (virtual/augmented/mixed reality) research. The book has 18 levels (chapters) and 11 bonus levels, and I am leveling up very slowly…
To better engage with the theories in the book, I also had to conduct 120+ hrs of “research” in D3 Rise of Necromancer. The book also repeatedly refers to BioShock, so I’ll have to get back to that at some point.
Below are my progressively updated notes. They probably don’t make much sense to anyone who has not read the book.
Level 1: N00bs
- What is a game: an activity that 1) requires at least one player, 2) has rules, and 3) has a win and/or lose condition.
- A game needs a clear objective so the player knows what the goal is. As a designer, you should be able to sum up a game’s objectives quickly and clearly.
- Games have two types of genres: story genre ( the type of story such as fantasy, sports) and game genre (the type of gameplay such as action, puzzle, shooter).
- Games are made by people with different skills: programmer, artist, designer, producer, tester, writer, product manager, creative manager, marketing, etc.
Level 2: Ideas
- Think what gamers want (good games). Make players feel something that they aren’t in the real world (powerful, smart, sneaky, bad, etc.).
- What’s the age of my audience? Kids always want what is made for an audience older than their own age group. Don’t make mistake by oversimplifying and talking down to younger audiences. Kids today are far smarter and way better gamers than we give them credit for.
- Despite some academic definition of fun (e.g., Marc LeBlanc’s sensation, fellowship, fantasy, discovery, narrative, expression, challenge, and submission), fun is completely subjective.
- You have no guarantee that your game idea is going to be fun. Theory of Un-fun: Remove all the un-fun, all that should be left is the fun. Don’t be afraid to kill bad ideas. If un-fun is ruining your game, kill the un-fun.
- Ideas are cheap; it’s how you use them that matters.
Level 3: Writing the story
- The most basic structure of a story:
- There is a hero with a desire (rescue a princess)
- The hero encounters an event that interferes with obtaining the desired, causing a problem.
- The hero tries to overcome the problem but his method fails
- Reversal of fortune. The fail causes more trouble.
- An even greater problem and one last problem (boss).
- The hero must resolve the final problem, and gain the object of desire.
- Every time someone plays a game, she creates a narrative. As a designer, you need to look at all the narratives possible and make them ALL fun.
- Designers help players to create the narrative. As each experience builds on the next, the goal is to create rising emotional states for the player. (Left 4 Dead uses AI that monitors players’ stress level – calculated using variables including health, skill, and location – then adjusts the items, enemies and the music.)
- Players’ narratives can end up quite different from the game’s story.
- The Triangle of Weirdness. Choose ONE from characters, activities, and world (to be weird). Choose more than one will risk alienating your audience.
- Make the story be in service of the gameplay and not the other way around (example: BioShock have non-mandatory collectible audiotapes that reveal deeper storying without intruding on the main story). Keep your stories lively and moving. Change in the plot or action every 15 minutes.
- Theme (e.g., love conquers all, eat or be eaten) can be more important than a story to a game.
- Determine how long the game should be.
- A game by Any Other Name – Still, choosing a name is the most important thing.
- Create characters your players care about. Give players time to bond with the characters. Make death matter.
- Game for kids: teach your players things without their even knowing it.
Level 4: Paperwork
- Making a game design document (GDD) – communication to the player, your team members, and to your publishing partners. The book provides templates in the bonus levels.
- Step 1: the One-Sheet – keep it interesting, informative, and short. Unique selling point – get readers excited about the features of a game without going into lengthy detail about them.
- Step 2: the Ten-Pager – people who finance your game will read this. Plenty of relevant visuals. Can be in PowerPoint. Tailor for audience type (the production team or marketing/executives). Includes the Title page, Game outline, Character, Gameplay, Game world, Game experience, Gameplay Mechanics, Enemies, Multiplayer/Bonus, and Monetization.
- Step 3: Gameplay progression – several different ways to start a game (start with nothing, several skills but needs to unlock, skills but no knowledge of how to use them, power but lose it after a fight, etc.).
- Step 4: The Beat Chart – describe level by level, as a “map” of the structure of the game.
- Step 5: GDD – Game designs are living things. GDD provides that launching pad from which to soar. Documents connect the producer, the designer, the artist, and the programmer.
- Step 6: Stay open-minded to ideas.
Level 5: Three Cs 1/3 – Character
- The Three Cs – Character, Camera, and Control – are probably the most important elements of gameplay.
- For the character design, an important rule is “Form follows function“. We want players to easily understand the personality of characters by their appearance (square->strong/dumb, circle->friendly, downward-pointing triangle->powerful or sinister depends on whether its body or face).
- Anything you can do to let the players customize their character furthers their feeling of ownership (including physique: eat too much junk food and get fat).
- Realistic or stylized design?
- Using all the parts to communicate information to the players (movement, appearance, inventory, weapons, etc.).
- Use a second character (playable or companion) and make players care about them. Use opposites attract where characters complement and contrast each other. Let the relationship develops early in the game.
- Differentiate characters so that each one has a weakness and a strength. Rock, Paper, Scissor (RPS) design.
- Use Non-player characters (NPCs) and how they are needed for the players to succeed (tools, access, gear, backstory, compliments,etc.). Given NPCs something to do when idle, which helps to enhance the environment. Have NPCs physically distinct in dress and body language. NPCs may even start a challenge with the player, mimicking a multi-player experience.
- Characters are determined by their metrics including height, speed, jump distance, attack distance, etc. Keep the metrics consistent.
- Walking is not gameplay. Avoid player travelling for long stretches sightseeing. Add moves, events, etc. Westerners are accustomed to reading things left to right, so have your characters walking to the left makes people feel “ill at ease”.
- Alternating fast and slow gameplay is interesting.
- Give your characters some short animation while they are not moving (idle). It’s the art of doing nothing.
- Jump is a complex movement. Think the mechanics of jumping. This reflects the decision on physics: real-world vs game physics. A certain fidelity to real life is necessary to sell real-world physics but break it to make the game fun.
- The book then elaborates on jumping and falling. The key messages are: 1) make the rules clear and consistent, 2) let players recover quickly (and continue with play).
- Shadow design varies. A simple drop shadow lets players understand where they’ll land.
Level 6: Three Cs 2/3 – Camera
- Nothing will cause players to stop playing your game faster than a poor camera
- Choosing the right camera view impacts the game design, control, and artwork.
- Static camera – fixed position, focal distance or field of view
- Scrolling camera – add advantages of a movement (better engagement) and reveal hidden stuff in a dramatic way. Parallax scrolling (world moves with camera), Forced scroll (players forced to keep up with camera movement or die), Mode 7, multi–plane camera (multiple layers of objects moving independently, used a lot by Disney. aka 2.5D)
- First person camera – watch DIMS (doom-induced motion sickness), influenced by the field of view. Remedies: high framerate, late stationary reference object, avoid whipping the camera too much, etc. No view, thus less bond with the characters.
- Third person camera – clear view of the character (‘s backside). Challenges on camera design:
- Camera movement: treat camera as a person, give it room to manoeuvre
- Sorting: move through a geometry with a collision. 1) detection radius to avoid collision or 2) turn object translucent, etc.
- Link to controls
- Camera flipping: camera bounces between objects.
- Obstruction: something gets between the camera and the player
- Position: camera strictly follows the player or laid back and follow freely?
- Who has the control of camera?
- Player control – 1) Allow players complete control over the following camera: players can get quickly disoriented, miss events, DIMS. 2) Free-look camera: simulating character’s head. limited range. Put cameras on hydraulics (or elastics). In certain context, simulate a piece of equipment such as binoculars or transition to the first-person view.
- No player control – less to worry about, designers focus on visible polygons or textures, no risk players missing key elements, must consider and match players movement, let players get out of it easily if the camera is blocked.
- Probably a hybrid of the previous two.
- Isometric camera – God view with no perspective adjustments. map 3d objects to 2d surfaces. In many cases, iso views are actually bi-metric view since only two axis pairs share the same angles. This is to accommodate 2:1 pixel for less pixelation. Elevation can be an issue for iso.
- Top-Down camera – sometimes top-down/side view.
- AR cameras – make sure things are scalable. try not to clutter up HUD elements (to avoid DIMS).
- Special case cameras – underwater or flying.
- Tunnel vision – player moving through tight environments like caves. use a rail camera to maintain the feeling of claustrophobia.
- Hollywood cheatsheet:
- Camera Shot guide – From extreme wide shot to Point-of-View shot and over-the-shoulder shot.
- Camera Angle Guide – Eye level, Worm’s-eye, Dutch tilt, etc.
- Camera Movement Guide – Arc, Dolly zoom, Pedestal, etc.
- Always point the camera to the objective. Even if players can’t see the objective, provide tools such as “detective mode” to pinpoints objectives, show an arrow/tag, turn obstruction transparent, etc.
- Multiplayer cameras.
Level 7: Three Cs 3/3 – Control
- Controls are universally applicable to every style of game.
- Away remember that humans are playing these games and we should avoid making the control itself too challenging (whilst the gameplay can still be).
- Assigning controls to different fingers based on what they are (not) good at.
- When designing for young players, keep the controls (button presses, etc.) simple.
- Do not go nuts with the uber complex controls.
- There are a lot of different controls on touchscreens. Use as few as possible to minimise confusing.
- A good designer will think about how the game is played in the real-world as well as in the game world. Button smashing -> Claw hands -> Occupational overuse syndrome.
- Never have a button do nothing when pressed:
- Play a “negative response”
- Make it clear that a button is inactive, then make a big deal when it is unlocked.
- Assign a redundant but related function (to engage players’ memory of that button)
- Map the moves to logical control locations helps immerse the player into the game world.
- Resembles that of other successful games.
- As the button is pressed, the action should happen (more or less promptly). Use long animation carefully and with a purpose.
- Most game controls are character-relative. Camera-relative ones don’t make much sense…
- Actuators and gyroscopes in a controller. Silent Hill used two actuators at different frequencies to simulate a heartbeat. Creepy… Tilt controls getting popular in mobile games. As the gyroscope is hidden within the mechanism of the controller, remind players that this function is available.
- Camera-based motion controllers – broad and mimic reality.
Level 8: Sign language: HUD and Icon design
- The heads-up display (visual screen overlay) is the most effective way of communicating with players.
- HUD is used for a wide spectrum of game elements to assist gameplay:
- in reverse: damage bar
- as a story device that represents the game’s narrative
- wait for health?
- replaced by first-person-style effects or a third-person view
- Targeting reticule
- stickiness (aim assist)
- colour/contrast consideration, zoomed-in mode
- Ammo gauge
- quick access to items
- how players arrange items?
- clear silhouettes
- permanent location or “magic box” that follows players through the game world
- arcade era
- online gaming leaderboards
- Positive messaging
- “Finish him!” keep players excited about their performance
- Positive re-enforcement for their actions that they don’t receive for their actions in everyday lives (or perhaps they are used to receiving those in real lives)
- big and flashy. make players feel good
- legible but not too big
- make it easy for the player to move and look at the map at the same time
- legend for icons to identify checkpoints, items, etc.
- indicate changes in elevation
- show current direction
- fog of war (clear as players moving through it).
- incorporate visual themes into the mini-map
- Context-sensitive prompts
- Icon of the button players have to use
- Characters thinking loud – players feel like the characters
- Use them as NPCs
- Indicates where to jump or travel
- Combat notification
- Clean screen
- Balance between HUD and “cinematic experience“
- Perhaps remove HUD altogether? Use sound, animation, and visual effects instead?
- Or temporary HUD
- Icons – keep your symbols simple
- Quick-time event (QTE) – not button mashing, use as an emotional effect? “Press x to save Jason”.
- Location of HUD
- Avoid middle of the screen or translucent/holographic
- Upper left used for most important information
- Bottom of the screen – watch out for clipping
- Should game continue while accessing inventory? Dead space didn’t – they wanted players not to be able to rely on the “gamey” mechanic.
- Title screen – movie poster, heroic pose, enigmatic (secret message, etc.), logo.
- Pause screen – order by most common selection. Think how players use them. Should be part of the game.
- Loading screen – Can be seamless by disguising the loading screen with slowly opening doors, long elevator rides, etc.
- Credit screen – give credit where credit is due.
Level 9: Level design
- The design of levels has different aspects:
- An environment or location where game play occurs
- Break up physical space based on a specific gameplay experience
- A unit of counting a player’s progression
- Rank of a player based on earned score, experience, or skills.
- Cliche video game themes: Outer space, Fire/ice, Dungeon/cavern/tomb, Factory, Jungle, Haunted house/graveyard, Pirate, Gritty urban, Space station, Sewer, etc.
- Name your game based on function, location, descriptive, punny.
- You can learn level design from theme parks like Disneyland. Theme parks are designed to move guests from one adventure to the net in the most effective way possible, like how a game should move players through different experiences.
- Build a map to help the team understand the connection between all levels.
- Use Foreshadowing to build anticipation.
- Design goals for each level: Escape/survive, Explore, Educate, or Provide a moral.
- Moral – Morals and consequences in games
- Use Beat chart to design and revise levels. Help to alternate between experiences and keep things fresh.
- Be smart with your art assets and repurpose them.
- Use landmarks to help players re-orient themselves. Never intentionally get players lost.
- If you need to make players go to a location more than twice, do something to make that experience different.
- Fingers is a way to make a world feel deeper and fuller without complex geometry and multiple paths. Every finger should have an award at the end.
- Alternating elevation and weather make an environment feel natural.
- If it looks as if players can go there, they should be able to.
- Don’t be bound by realism, especially when it contradicts game experience/camera, control
- Design your games between “big moments” and “small moments” so players can get a rest between.
- Pace out your game so you can control the overall length of a game.
Level 10: The elements of combat
- Many video games are violent – easy to introduce actions.
- graphic, dramatic, visceral
- quick response feedback
- pleasure principle – the instinctual seeking of pleasure and avoiding of pain
- other human interactions such as conversation, romance, humour, etc are often hard to re-create
- Think of a unique combat style for your character
- distance – close-range, medium, long, area effects/bomb
- elevations – standing position, low, hight, aerial
- Build an attack matrix so you can make every character unique – this applies to weapons too.
- When player press a button, waste no time on prolonged animations and perform actions immediately.
- Use particle and visual effects to make attacks feel more dynamic and rewarding.
- Provide super move for the big finish. The enemy should react to wherever he’s been hit.
- Use camera movements to show the impact of strikes. – People want to play games that make them look cool
- Weapon often defines a character.
- Don’t forget impromptu weapons – chairs, pipes, etc.
- Use lock-on system to help player aim
- Allow players to defend themselves but don’t turn a defensive manoeuvre into a disadvantage
- Make characters’ appearance reflecting statuses such as health or new ability.
- A few tricks to scare players on Page 292. – Sound is the scariest thing there is. Use these principles in a positive way for trauma therapy, etc.
- Don’t let the enemy have all the fun. Strive to enable the players.
- Design death carefully so to keep players playing. Don’t give players a good reason to stop.
- You can create conflict without combat: timer, speed, limited space, moral dilemmas, cost, etc.
Level 11: Enemy design
- Form follows function
- Size up the enemy based on the size of character – let it influence health and reaction to attacks
- Enemy behaviour: patroller, chaser, shooter, guard, flyer, bomber, burrower, teleporter, blocker, doppelganger and they may swap between these.
- The speed, size, and strength of an enemy are inversely proportional.
- Give your enemies different movement patterns will make them feel more realistic
- Enemy introduction – tell players they’ve encountered something new, exciting and dangerous.
- Fighting enemies is supposed to be fun and not avoided.
- Reasons to fight an enemy: loot, they block path, they have the key, need their power, they make fun of you, fun to fight.
- Enemy offers you a chance to really flex your creative muscle as a designer
- start with your theme or story
- tie two of them together
- be economical with your enemies
- make them look like an enemy
- Make players hate their enemies
- Non-enemy enemies
- Design boss fights
- Relates the boss to the hero
- physical adversary or mental adversary
- attack pattern to follow/understand
- adaptive difficulty?
- the last strike needs to be delivered by the player
Level 12: Mechanics
- Flow channel that keep players between anxiety and boredom (again)
- Orchestrate gameplay element:
- start your hero moving through the world with simple movement
- start with one mechanic and repeat a couple of times
- add a second mechanic then combine with the first
- make things exciting with a hazard
- now come the enemies
- combine the enemies with hazards for more excitement
- add variety to the above
- checkpoints that allow players to save progress, take a break, or reassess choices of equipment, route, etc.
- Puzzles and mini-games! simple and short
Level 13: Power up
- Immediate effect
- Four categories: defensive, offensive, movement, and game changers
- Game changers alter the dynamics of the gameplay and player’s interaction with the game in a significant manner (pac man)
- Love the player. Give them help when they struggle. Dynamic difficulty balancing, rubberbanding, etc.
- Never underestimate the greed of the players (steam-powered chicken, i want one!)
- Rewards, never have a victory without a reward (expose that early and reward asap).
- High score, achievements, MoneyMoneyMoney. – give player the “delicious agony over what to buy”!
- Bonus features: costume changes, alternative models/modes, downloadable content, new levels, multiplayer, different endings, big head, etc.
Level 14: Multiplayer
- Styles: competitive, cooperative, conjugate.
- MMORPGs – buffing, character customization, chattering, crafting, economy, grinding (slow progression repetitive), open world structure, trading/auctions.
- Determine the impact of other players
Level 15: Monetization
- Ways to make cash
- pay to play
- in-game advert
- virtual goods – the association of real-world money and game-world efforts/skills
- downloadable content
- season pass
- Provide alternative ways to earn
- Warn players that game costs to play
- Provide discounts
- Let players back out of the pay option
Level 16: Music
- Music requires a lot of work and coordination between many members of a team so don’t leave it until the end.
- Think what you can do with music creatively
- Be careful of the licence
- Break music into themes that play when a certain situation arises. Mystery, warning, combat, chase, victory, walking – Always make the music more exciting than the action on-screen.
- Use silence – a sound not a sound.
- Use sound for movement, attacks, impacts, weapons, hit reactions, vocal cues, death, success, etc.
- Timing! synchronicity with events on screen.
- Designate sounds into three categories: local (close to the source), distant (far from the source), and priority (regardless of source).
- Some games use sound/music as part of the gameplay
Level 17: Cutscene
- Allow the game world/characters to be shown in a way that may not be reproducible in the game engine.
- you can create more emotion in a cutscene
- pre-rendered scenes look awesome
- you may take control away from the player so they can enjoy it
- You can have cutscenes either non-interactive, fully interactive or between.
- How to write a screenplay in eight steps:
- Outline your story
- Break the story down by scene and determine which characters are in each scene (location, etc.)
- Determine which scenes are going to be cutscenes vs being told through gameplay
- Write your scenes and dialogue
- Write your script in official screenplay format (pseudo code)
- Read your dialogue
- Let it simmer for a day or two
- Prepare your script for voice actors using a spreadsheet programme.
- Find your voice – get voice director, actors, etc.
Level 18: Publish your game
- Demo > GDD > pitch presentation > pitch outline
- Pitch to the right audience
- Build games using either horizontal layer– or vertical slice-style production
- If something doesn’t work, throw it out – same for the sequel.
[end of post]