A test report

When I don’t have much project work to do, I do a lot of different things that add certain value to my skill as a tester. I read blogs, watch podcasts and videos. Sometime I pick up an open source project from sites like http://sourceforge.net and test it. I prepare test report and analyze it myself. In this process I learn many things. On one such occasion, I picked up a product, tested it and prepared a test report. You might like to see the test report. Below it is:

                Test Report: The Number Race 2.3.4
                                            (Author: Prabhat Nayak)

Test Mission:
The mission is to test “The Number Race” application which has been awarded as the project of the month for November by sourceforge.net and find out some important problems in a session of two hours. The quality criteria that are going to be covered in this session are “Usability” and “Capability”. I’ll not prepare any document before my actual testing. It will be more of exploratory testing and keeping note as I test.

Test Environment:
Operating System: Windows XP Professional, Service Pack 3
Application Version: NumberRace 2.3.4

Test Areas:
1. Installer
2. Select Player
3. Menu
4. Level 1 Game
5. Uninstaller

Test Approach:
I downloaded the application from http://sourceforge.net/projects/numberrace/. I divided my two hours of time into 3 phases as below:

Phase1 (30 mins): This phase was dedicated to gathering information about the application. This included who are the intended users, what is the purpose of building the application and how to use the application.

Phase 2 (60 mins): This phase involved actual learning/testing efforts of the application. I started my exploration from installing the application. Then I created/selected a player and played a level. I then uninstalled the application. During all these activities I took notes of my observations.

Phase 3 (30 mins): In this phase, I prepared the final test report on my findings.

Observations:
1. Double clicking the installer invokes the language selection window. In this window appears the “how to start” instruction which could hardly be read. Ironically this is the only place which shows this information. Refer to “Starting_Point.JPG”.

2. Installation-step-5: If "Create shortcuts in the Start-Menu" checkbox is unticked, "Create additional shortcuts on the desktop" option gets disabled. In this way one cannot create only the desktop shortcut without creating Start-Menu shortcut.

3. Installation-step-6: After installation gets completed, Instead of button name "Quit", "Finish" could have been more meaningful.

4. After launching the application, "No language pack installed" dialog box appears. It asks to download the needed language pack without giving any information on where to download it. Please refer to “Language pack.JPG”.

5. The start page doesn’t clearly display the starting point by clicking which the game could be started.

6. While creating a player, providing invalid information (other than leaving it blank) and clicking the “Enter” button doesn’t result in creating a player but it doesn’t show any error/warning message either. Message is shown only in case a field is left blank.

7. Age filed is validated only against non-numeric data. It accepts 0 and all negative numbers.

8. Pre specified “Level” for an existing player cannot be modified.

9. Very less information on “how to play a level” is displayed prior to start the level. It doesn’t suffice in understanding the level.

10. After a move is finished, it takes few seconds for the sound to play and hence delays the process of going to the next move.

11. After completion of a level, the player still remains in the same level. It should get alleviated to the next level.

12. On clicking question mark icon on Select Player window, a window is invoked. This window is blank and reveals no information. Either it should display some information or it should be removed. Please refer to “Questionmark.JPG”.

13. Clicking “Quit” button terminates the application without displaying a confirmation message.

14. Uninstalling from the folder hierarchy (for eg. C:\Program Files\NumberRace-2.3.4\Uninstaller) leaves some empty folders.

Issues:
Issue #1:
Summary: 000101: Unticking "Create shortcuts in the Start-Menu" checkbox disables "Create additional shortcuts on the desktop" option
Product: NumberRace 2.3.4
Description: In step 5 of installation, when "Create shortcuts in the Start-Menu" checkbox is ticked, "Create additional shortcuts on the desktop" checkbox gets disabled. The only way to get a desktop shortcut is to allow the installation to create Start menu shortcut.
Steps: 1. Double click the installer
2. Go to step 5 of installation
3. Untick the "Create shortcuts in the Start-Menu" checkbox
Impact: A user can never have a desktop shortcut without having a start menu short cut. Start menu shortcut is unwanted for those who like to launch the application from desktop shortcut. They might not like this feature.
Attachements: Please refer to image “Shortcut_Issue.JPG”
=============================================

Issue #2:
Summary: 000102: Age field accepts 0 and negative numbers
Product: NumberRace 2.3.4
Description: Age field in Select Player window takes 0 as well as all negative numbers which are invalid. It is only validated against non-numeric data
Steps: 1. Go to Select Player window
2. Fill all the fields excluding Age field with valid data. Provide 0 or any negative number on Age field.
3. Click on Enter button
Impact: Age of a player is very important when this application is used to track the real progress in terms of capability vs. age of a child. In that case allowing wrong data as age can mislead the assessment.
Attachements:
=============================================
Issue #3:
Summary: 000103: Pre-specified Initial Level for an existing player cannot be modified
Product: NumberRace 2.3.4
Description: Initial Level field in Select Player window shows which level a particular player is in. According to the level selected, the player gets a game of that level to play. An existing player cannot change his level even if he has finished the same level and eligible to graduate to the next level.
Steps: 1. Go to Select Player window
2. Fill all the fields with valid data.
3. Select any level (1-14) from the Initial Level list.
4. Click on Enter button
5. Click on Modify button
6. The Initial Level list is disabled
Impact: An existing player cannot graduate to the next level by selecting a higher level even if he has completed the specified level. In a way to do it, either he has to create another player or delete the existing one and recreate a player with the same details.
Attachements: