How Game Testing Increases User Engagement: Secrets from QA Program Manager
User engagement defines a successful game. To explain how to improve this aspect, we held a webinar hosted by Ivan Kovalchuk, Program Manager, and QA engineer with more than 5 years of experience in game testing in QATestLab. While you can watch the full record of the webinar on our YouTube channel, here are extracted tips and tricks of it. This information will be especially useful for product owners, QA engineers, and just curious minds that want to know all the secrets behind successful game testing.
Tips to Boost Game testing from QA Program Manager
The solid foundation of your game success is security testing.
“Living in today’s data-driven world, security issues like data privacy are paramount things for absolutely all products, not only for games. If you are offering totally secure products with high data protection features, the number of your users will increase, without mentioning how much money you can save having security at a high level.” — highlighted Ivan.
How to ensure the highest security during testing game products? Ivan shared several points for achieving that:
1. All critical system local directories and files should be encrypted. Moreover, they need to be hidden or unavailable at all.
Ivan: “In this case, we’re checking local directories on the physical device by file explorer. The easiest way for Android devices — opening way /android/data/%android_application_id%/cache/… and find the required files/folders. In edge cases, we need to use root access and special software/hardware tools for receiving extra access to some hidden folders.”
2. Checking abusing coins/gems by simply changing specific files in local game saves.
To avoid the possibilities of abusing coins/gems, and other hackers-like manipulations, a QA team should analyze the savings model and local storage options by Android SDK, iFunbox, iTools, and other tools. Scanning and thoroughly checking all available local folders/files by external tools would be also useful. Another issue to check is the sharing savings of high-level accounts (Coins, XP, resources, rank) into other devices with beginner accounts.
“…knowing that the game is operating with the same rules for absolutely everyone, that every player is equal, is a great motivation to play such a game” — added the host of the webinar Olga.
3. Testing the possibility of changing options for the time/date setting for receiving rewards and special offers/events.
Such issues can be detected by testing games on different platforms for all time zones with custom and automatic time settings. Another approach is to change server time by using special hardware/software tools. Finally, we need to ensure that changing the time/date setting for receiving rewards and special offers/events will not work. And gaming experience will be clear and fair to any user.
4. Checking external access to the user’s personal data.
There’s no need to explain, but the issue is not just about the direct defense, it’s rather about using social network accounts in the game and getting access to them by the game’s API. Such kind of vulnerability is not so popular with hackers but sometimes it occurs.
To detect such holes would it can be useful to create a pool of test accounts in different social networks from different countries (by VPN) and use a multi-attempt approach for users’ critical data access. For more detailed API checking Ivan recommends using tools with get/post access features (for example, Swagger, Postman, etc). They might help to check the most common vulnerabilities in a very short period of time.
5. Check whether client-server transactions/requests are editable by the user.
Abusing of in-game currency, cheating levels, receiving new skins, changing purchased items quantity, and many more critical things occur in plenty of games. Some of such issues are deeply hidden, but the great majority could be fully detected and analyzed. Ignoring such kinds of vulnerabilities might cause extremely low ARPU levels and real user outflow.
“We can use such popular traffic logging tools as Fiddler or Charles for catching/monitoring between devices and the server/Internet. This includes requests, responses, etc. (can contain the cookies and cache info). Moreover, we need to test behavior after editing such requests for abusing resources or reaching/cheating levels.” — said Ivan.
In-App Purchases Testing: Critical Checks for Game Testing
In-app purchases testing is the second paramount thing in game testing that affects user engagement and the success of the game to the greatest extent. Here are the secrets shared by Ivan Kovalchuk on the webinar on game testing:
1. All the items in the Game Store should be available.
To properly solve this issue, for the QA team, it is mandatory to conduct manual tapping/clicking all available items in-store and checking redirections into the game store from all required places in game.
“We prefer to test purchases not just by sandbox features, we’d rather do such checks with the real money for priority countries according to customer needs.” — emphasized Ivan.
2. Every item in the Game Store should appear with proper currency according to the user’s country of origin.
To ensure this, it is mandatory to conduct a thorough check of all items in the game store by using accounts of required countries according to developers’ needs. By the way, most accounts need to be created via VPN services for covering all needed territories. The number of purchased items in inventory must be the same as the number selected in the Game Store.
3. Spent costs for purchased items should match the price in the Game Store.
“In this case, we use different combinations of store items vs quantities and make purchases with sandbox features, gift cards, as well as purchases for the real money. As usual, we check all popular combinations or required items/offers according to the needs of the developer.” — commented Ivan.
4. Funds can not be withdrawn from the account when the purchasing process wasn’t finished or ended unsuccessfully (without gaining any item from in-game store).
“…we can use a detailed interruptions checklist in this case as well as VPN checks or even mix them for more stressful testing. Also, it would be good to check different types of credit/debit cards with edge payment restrictions and different balances on the cards. Everything depends on the requirements and priorities of developers.” — that’s what Ivan recommended for solving this in-app purchase issue.
More information on these issues as well as other questions on game testing can be found in the full record of our webinar.