Reviewer's Comments
Each review includes a "Reviewer's Comments" section. This allows the reviewer to give their subjective opinion and also allows for explanation of important aspects of the review. Aside from this section, reviewers strive to be totally subjective and evaluate apps based on the following guidelines.
Reviewer's Guidelines
The following is an example of some of the criteria and the guidelines our reviewers use in their reviews. With the wide variety of app and the fact that we are human it is impossible to be totally objective, but we try very hard to apply the same standards to all of our reviews. Our reviewer's select the criteria from the predefine list. The scores and guidelines are defined below. We may from time-to-time make adjustments to the criteria and weights based on our experiences and the input from users like you. Please feel free to contact us if you have input or comments.
|
Criteria Pick List
|
Criteria Description
|
Criteria Value
|
Grading Guides
|
| Alternative Languages | Does the app support alternate languages? |
2
|
English Only = 1 point One to four = 5 points Five or more = 10 points |
| Audio Quality | Are the sound effects clear understandable and consistent in quality and volume? |
3
|
Score according to sound quality. Reduce score if audio fidelity is unclear, hard to hear, or inconsistent in level. Consider the intended audience and use. Be as objective as possible. Reduced scores will depend on the type of app and intended use. |
| Audio Usefulness | Are sounds and music relevant and appropriate? |
3
|
Score according to how the sound matches the application needs. Reduce score if sound is confusing or seems out of place. Consider the intended audience and use. Be as objective as possible. Reduced scores will depend on the type of app and intended use. |
| Bugs - | Does the app work as expected, without bugs |
5
|
Test as many aspects as possible. If app crashes or hangs in one area or function reduce score. If multiple problems and app crashes or hangs in different areas score low. Also reduce score if app stalls and doesn’t give user indication that something is going on. Consider the intended audience and use. Be as objective as possible. Reduced scores will depend on the type of app and intended use. |
| Clean Installation | Did the app install easily and cleanly? |
5
|
After load did app run first time? If not reduce score. If iPhone has to be rebooted before use reduce score by 3 points. Consider the intended audience and use. Be as objective as possible. Reduced scores will depend on the type of app and intended use. |
| Configuration Options | Does app offer configuration options to allow customization? |
5
|
If applicable, the app should offer user ability to customize screens, layouts, audio video effects….. Examples:
|
| Controls | Does app offer adequate controls for purpose? |
5
|
Are there sufficient controls for app type. Are controls easy to get to (on main screen or logically placed). Reduce scores will depend on app type and features. |
| Easy Configuration | Was initial configuration simple and easy to understand? |
5
|
When first starting app should offer initial instructions on screen or provide easy access to help. If the app requires custom configuration before use it should be obvious. Reduce scores will depend on app type and features. |
| Entertainment Value | Is the app entertaining? |
5
|
If app is entertainment app, is it? This can be very subjective so reviewer should consider the intended audience. Reduce scores will depend on app type and features. |
| Feature Set - | Does the app have a full feature set for the purpose? |
5
|
If app is designed to serve a general purpose does it have enough features for the purpose? Examples:
|
| File Import Export | Dose app allow files to be moved between iPhone and PC/MAC? |
5
|
If app uses or produces files can they be moved on and off the iPhone? Reduce score if it makes sense to do this, but not supported or hard to accomplish. |
| File Format Support | Does the app support common file formats? |
5
|
If app needs to save files such as images or art does it offer multiple formats. If it’s designed for a specific task this may not matter, but if it’s designed to be general it may be desirable to save or open different formats of similar files. Reduce scores will depend on app type and features |
| Game - Control | Is game control precise and accurate to the touch (pinpoint of the touch) type of game? |
5
|
Game interface should be accurate. If it’s hard to hit the right spot or touches seem off center reduce score. If it uses accelerometer it should be responsive without being too touchy. Accelerometer apps should have some sort of calibration. |
| Game - Entertainment Value | With respect to game type (action, FPS, board…) is the game engaging and entertaining? |
5
|
If app is entertainment app, is it entertaining? This can be very subjective so reviewer should consider the intended audience |
| Game - Learning Curve | Can you pick it up and play or is there a learning curve? |
5
|
How hard is it to learn. It’s OK to have a more difficult learning curve but if it does it should offer starter levels or initial help. Consider the intended audience and use. Be as objective as possible. Reduced scores will depend on the type of app and intended use. |
| Game - Options | Does app offer options for game play to allow variations in how the game is played (character selection, courses, …)? |
5
|
Does the game offer enough characrter selections, levels, courses, etc to keep the user engaged for a while? Consider the audience and game type. Consider the intended audience and use. Be as objective as possible. Reduced scores will depend on the type of app and intended use. |
| Game - Pace | Is the pace controllable, including options for stop/pause/restart? |
5
|
If game play is extended can it be paused without starting over. At least after major sections. Can you control the pace by speed or difficulty? Consider the intended audience and use. Be as objective as possible. Reduced scores will depend on the type of app and intended use. |
| Game - Save/Replay | Does the app have option for saving games or activities in progress for later replay? |
5
|
If game play is extended can it be stopped and restarted at a later time. Consider the intended audience and use. Be as objective as possible. Reduced scores will depend on the type of app and intended use. |
| Game - Skill Levels | Are there sufficient settings to allow the game to be altered to suit user skill and level? |
5
|
Can levels be set for different users. Consider the audience and game type. Consider the intended audience and use. Be as objective as possible. Reduced scores will depend on the type of app and intended use. |
| Graphics Quality | Are the graphics of good quality and resolution; do images load within reasonable timeframe? |
5
|
If the app uses graphics, including icons buttons, etc are they good quality, and clear. If it uses large graphics are they sized correctly and do they load in a reasonable time? Reduce scores will depend on app type and features. |
| Graphics Usefulness | Are the graphics meaningful and useful? |
5
|
Are the graphics understandable and suitable for the purpose? Reduce scores will depend on app type and features. |
| Help - | Does app have built in help that is really helpful? (Direct access in app, access via web, or none. |
5
|
Does the app have built in help? Is the help easy to get to from the point it is needed? If web based help does it force you to exit app to browser to see the help? Reduce scores will depend on app type and features |
| Intuitive Interface | Is the interface intuitive (icons, menus, and directional symbols) for average users and does it adhere to good iPhone design principles? |
5
|
Does the interface make sense for the type of app. Do you have to guess where to go for desired features? Is it clear and concise? Reduce scores will depend on app type and features |
| iPhone Interface | Does the interface follow good iPhone GUI design practices? |
5
|
Did the developer use an interface design that makes sense for the iPhone? Did they deviate from Apple’s intended design guidelines? Reduce scores will depend on app type and features |
| Learning Curve | Is app easy to lean and use? |
5
|
How hard is it to learn. It’s OK to have a more difficult learning curve, but if it does it should offer initial help. Consider the intended audience and use. Be as objective as possible. Reduced scores will depend on the type of app and intended use. |
| Link to Desktop App | If app can link to desktop app is configuration and linking to desktop app easy? |
5
|
If the app links to a desktop app is it simple to setup and use? Consider the intended audience and use. Be as objective as possible. Reduced scores will depend on the type of app and intended use. |
| Load Time - | Does the app load quickly? ( ready to use in less than 10 seconds on 3GS) |
3
|
Does it load in a reasonable time? Apps where users expect large graphics are expected to take a little longer to load. Does it indicate that this is the case? Consider the intended audience and use. Be as objective as possible. Reduced scores will depend on the type of app and intended use. |
| Navigation | Is the app navigation logical and easy to use? |
4
|
If there are multiple screens (help, setup…) is it logical as far as how and when to get to them? Consider the intended audience and use. Be as objective as possible. Reduced scores will depend on the type of app and intended use. |
| Online Account Required | If an online account is needed, is it easy to setup? |
5
|
If an online account is required is there a clear link to site. When you get to the site do you know exactly where to go and what to do? Consider the intended audience and use. Be as objective as possible. Reduced scores will depend on the type of app and intended use. |
| Readability | Legible text and print size that is appropriate for the intended audience. Are fonts and interface elements easy to see and read? |
5
|
Is text on the screen readable? If the app is intended for a lot of reading (like an eReader) can you adjust fonts and spacing? Consider the intended audience and use. Be as objective as possible. Reduced scores will depend on the type of app and intended use. |
| Regular Updates | Does vendor appear to keep app up-to-date? |
3
|
Have there been revisions to fix bugs (good thing)? Any indication of upcoming updates (good thing)? If v1.0 how long since release. It is possible they got it right the first time but if it has issues in other areas it may be abandon. Check web site to see if there active communications from developer. Consider the intended audience and use. Be as objective as possible. Reduced scores will depend on the type of app and intended use. |
| Game Replay Value | Will user feel compelled to play same level over. |
5
|
Would the user want to play again or get bored after a few times. This can be subjective so think about intended audience. Consider the intended audience and use. Be as objective as possible. Reduced scores will depend on the type of app and intended use. |
| Screen Layouts | Are screen layouts logical and well-designed? |
5
|
Are the screens laid out well? Graphics, proportions, spacing, useablilty. Consider the intended audience and use. Be as objective as possible. Reduced scores will depend on the type of app and intended use. |
| Support Website | Does the developer offer a comprehensive support site? |
4
|
Go to app web site. Does it look current? Does it add value? Consider the intended audience and use. Be as objective as possible. Reduced scores will depend on the type of app and intended use. |
| Value Price | Does the app offer enough value ( features, functionality) for money or to be useful. |
5
|
If free not really applicable. If paid would user get their moneys worth? Consider the intended audience and use. Be as objective as possible. Reduced scores will depend on the type of app and intended use. |
| Video Quality | Are video effects clear and easy to see? |
5
|
If the app uses video as part of the GUI is it good quality? Is it fast enough? Consider the intended audience and use. Be as objective as possible. Reduced scores will depend on the type of app and intended use. |
| Video Usefulness | Are video effects used appropriately? |
5
|
If the app uses video as part of the GUI does it add value to the experience? If the app uses video as part of the GUI Consider the intended audience and use. Be as objective as possible. Reduced scores will depend on the type of app and intended use. |
