Smartphones as Controllers

the smartphone controllers are simultaneously one of the biggest challenges in developing games for airconsole and also the aspect with the most potential for true innovation when developing an airconsole game, the smartphone controls should be an integral part of early gameplay prototyping, since some mechanics that work with a gamepad will often feel subpar when controlled with a smartphone while we are proud of the over 100 games the airconsole store currently offers, we believe that we have barely scratched the surface of what is possible in regards to innovative, intuitive and fun smartphone controlled local multiplayer games main takeaways use different controller views / layouts for different situations label buttons / interactive areas with their actual function make all buttons / interactive areas as big as possible make use of personalized controllers for individual players do not use d pads and virtual joysticks play existing airconsole games to see which control schemes feel good and which don’t feel free to ask the airconsole team or our developer community on discord for recommendations for a specific type of game or control scheme you are interested in smartphones are not gamepads when designing a smartphone controller layout, the first and most important thing to realize and remember is that a smartphone screen is not a regular gamepad / xbox controller and should not be treated as such this provides some challenges, but also has its advantages pro individual layouts the controller screen can be adjusted to meet any ingame situation and mechanic it needs to whether you need it to only display a single big button or you want to let the player play an entire minigame on it instead of having buttons (or other interactive areas) labelled with “a”, “b” or “x”, you can label all touch areas directly with their function and don’t need to include captions for them on the main screen pro personalized information every individual player can have their own controller view this can be used for something as simple as displaying the player color, but you can also convey secret information (cards, items, secret objectives) or design asymmetrical input schemes where different players have different roles and input options pro input variety virtual buttons are not the only way to register input from a smartphone besides button taps, your game input can also use swiping and the phone's gyroscope, or even its camera, speakers or microphone everything that works in a mobile browser is possible on airconsole con no tactile feedback the most significant disadvantage smartphone controllers have over regular gamepads is the lack of tactile feedback a player’s finger may slip over the screen while they are not looking down, and the player might not realize that their finger is no longer on the button or interactive area, which leads to frustration to combat this, interactive areas / buttons on airconsole controllers should be made as big as possible, reducing the chance of players tapping empty space and growing frustrated views what are views? what we call views are different controller layouts for different ingame situations in the pictures below, you can see the “ingame” and “pause” views for dust squad tanks for nothing views can be updated for more specific and/or frequently changing situations too three examples from dust squad where the view shows that the player’s weapon is currently on cooldown the player is close to hero exclusive content and can buy a subscription here the player has died and is waiting for respawn how do i change views? we leave the exact implementation to you, but we recommend using custom device states and not messages to change your views we also recommend using the airconsole view manager with it you can add the ‘view’ class to div containers and easily show one of them and hide all others using screen estate input fields (buttons, swipe areas etc ) should always be as big as possible, so that they are hard to miss reduce the number of different input fields by only showing the ones that are currently needed and updating the controller view whenever necessary bad good visuals the controller’s visual design should fit the ingame graphics and big screen ui, adding up to an aesthetically pleasing whole picture when it comes to developing a ui art style with smartphone controllers in mind, it is helpful to use elements that can be stretched instead of relying on exact angles or aspect ratios this can make the implementation for different phone screen resolutions with html/css easier joysticks & d pads virtual joysticks and movement d pads do not work for airconsole we have several games that use them, but free character movement with analogue sticks or directional buttons will always feel subpar on a smartphone controller in comparison to a gamepad this means that you cannot rely on including “character movement” in your game design without giving serious thought to how exactly it will work with a smartphone how character movement should work instead depends heavily on the game and there is no universal answer for how to solve this issue this limitation should be considered in your game design from the very beginning haptic feedback give you players more control over their actions by letting them feel what they are doing with haptic feedbacks be it direct feedback of the controls just on the controller side or vibrations triggered by ingame events happening on the screen the controller app of airconsole for ios and android gives you more control over vibrations on the controller devices with the vibrate function of the airconsole api you can play several vibration primitives and chain them together to form a more complex pattern use the composition vibration interface to play vibration primitives by passing the vibrationoptions to the vibrate function airconsole vibrate({ type airconsole vibrate type composition, value \[ { primitive airconsole vibrate primitive thud, scale 0 8, } ] }) the above code is playing a vibration pattern that emulates a thud with a strength of 0 8 you can find all the available primitives in our api documentation to be able to create more complex pattern you can chain two or more primitives together to create a new pattern that can be executed with one call airconsole vibrate({ type airconsole vibrate type composition, value \[ { primitive airconsole vibrate primitive quick fall, scale 0 8, }, { primitive airconsole vibrate primitive tick, scale 0 9, delay 10, }, ] }) the example above plays a quick fall vibration followed by a short tick vibration effect with a delay of 10ms after the quick fall has completed please note that executing vibrations also drains the battery also make sure that you don't over do it an numb you players hands