Team project page: http://code.google.com/p/hale-aloha-cli-tiger/
Today, we are conducting a peer technical review, taking on three different perspectives:
User's perspective: Does the system accomplish a useful task?
Installer's perspective: Can an external user successfully install and use the system?
Developer's perspective: Can an external developer successfully understand and enhance the system?
First, it should be understood that all software projects are never completed. Rather, they are continuously being enhanced and enriched. Thus, the issues pointed out here are relevant only up to revision 67 of the project.
User's Perspective:
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| $ java -jar build/jar/hale-aloha-cli-tiger.jar | |
| Successfully connected to the Hale Aloha Wattdepot Server | |
| > current-power Ilima-A | |
| Current power for Ilima-A as of 2011-12-02 06:49:59 is 3.91 kilowatts. | |
| > daily-energy Mokihana 2011-11-05 | |
| Date must be before today. | |
| > daily-energy Mokihana 2011-12-01 | |
| 2011-12-01T00:00:00.000-10:00 | |
| 2011-12-01T23:59:59.999-10:00 | |
| Mokihana's energy consumption for 2011-12-01 was: 659 kWh. | |
| > daily-energy Mokihana 2011-11-01 | |
| Exception in thread "main" org.wattdepot.client.BadXmlException: 400: Range extends beyond sensor data, startTime 2011-11-01T00:00:00.000-10:00, endTime 2011-11-01T23:59:59.999-10:00: Request: GET http://server.wattdepot.org:8190/wattdepot/sources/Mokihana/energy/?startTime=2011-11-01T00:00:00.000-10:00&endTime=2011-11-01T23:59:59.999-10:00 | |
| at org.wattdepot.client.WattDepotClient.getEnergy(WattDepotClient.java:762) | |
| at org.wattdepot.client.WattDepotClient.getEnergyValue(WattDepotClient.java:810) | |
| at org.wattdepot.client.WattDepotClient.getEnergyConsumed(WattDepotClient.java:857) | |
| at edu.hawaii.halealohacli.command.DailyEnergy.run(DailyEnergy.java:85) | |
| at edu.hawaii.halealohacli.processor.CommandProcessor.chooseModule(CommandProcessor.java:45) | |
| at edu.hawaii.halealohacli.processor.CommandProcessor.run(CommandProcessor.java:71) | |
| at edu.hawaii.halealohacli.Main.main(Main.java:38) | |
| > energy-since Lehua-E 2011-11-01 | |
| org.wattdepot.client.BadXmlException: 400: Range extends beyond sensor data, startTime 2011-11-01T00:00:00.000-10:00, endTime 2011-12-02T06:57:14.561-10:00: Request: GET http://server.wattdepot.org:8190/wattdepot/sources/Lehua-E/energy/?startTime=2011-11-01T00:00:00-10:00&endTime=2011-12-02T06:57:14.561-10:00 | |
| at org.wattdepot.client.WattDepotClient.getEnergy(WattDepotClient.java:762) | |
| at org.wattdepot.client.WattDepotClient.getEnergyValue(WattDepotClient.java:810) | |
| at org.wattdepot.client.WattDepotClient.getEnergyConsumed(WattDepotClient.java:857) | |
| at edu.hawaii.halealohacli.command.EnergySince.getEnergyConsumed(EnergySince.java:100) | |
| at edu.hawaii.halealohacli.command.EnergySince.run(EnergySince.java:72) | |
| at edu.hawaii.halealohacli.processor.CommandProcessor.chooseModule(CommandProcessor.java:49) | |
| at edu.hawaii.halealohacli.processor.CommandProcessor.run(CommandProcessor.java:71) | |
| at edu.hawaii.halealohacli.Main.main(Main.java:38) | |
| Total energy consumption by Lehua-E from 2011-11-01 00:00:00 to 2011-12-02 06:57:14 is: 0.0 kWh. | |
| > energy-since Lehua-E 2011-12-01 | |
| Total energy consumption by Lehua-E from 2011-12-01 00:00:00 to 2011-12-02 06:57:14 is: 158.4 kWh. | |
| > rank-towers 2011-11-01 2011-11-09 | |
| Date must be before today. | |
| > rank-towers 2011-11-01 2011-12-01 | |
| Exception in thread "main" org.wattdepot.client.BadXmlException: 400: Range extends beyond sensor data, startTime 2011-11-01T00:00:00.000-10:00, endTime 2011-12-01T23:59:59.999-10:00: Request: GET http://server.wattdepot.org:8190/wattdepot/sources/Ilima/energy/?startTime=2011-11-01T00:00:00.000-10:00&endTime=2011-12-01T23:59:59.999-10:00 | |
| at org.wattdepot.client.WattDepotClient.getEnergy(WattDepotClient.java:762) | |
| at org.wattdepot.client.WattDepotClient.getEnergyValue(WattDepotClient.java:810) | |
| at org.wattdepot.client.WattDepotClient.getEnergyConsumed(WattDepotClient.java:857) | |
| at edu.hawaii.halealohacli.command.RankTowers.run(RankTowers.java:106) | |
| at edu.hawaii.halealohacli.processor.CommandProcessor.chooseModule(CommandProcessor.java:53) | |
| at edu.hawaii.halealohacli.processor.CommandProcessor.run(CommandProcessor.java:71) | |
| at edu.hawaii.halealohacli.Main.main(Main.java:38) | |
| > rank-towers 2011-11-23 2011-12-01 | |
| For the interval 2011-11-23 to 2011-12-01, energy consumption by tower was: | |
| Mokihana 5289 kWh | |
| Lehua 5290 kWh | |
| Ilima 5426 kWh | |
| Lokelani 6219 kWh | |
| > quit |
From the user's point of view (see sample executions above) the program could use a bit of polishing on its command handling and error reporting. For example, the command "current-power" works correctly, while the other three commands, "daily-energy", "energy-since", and "rank-towers", suffers from date handling problems. Computational-wise, they worked correctly. However, the date handler code suffers from a "single-month view" problem for date comparison, and couldn't handle out-of-range date gracefully.
Installer's Perspective:
From the installer's perspective, the project site has the basic description and user's guide needed to successfully install the system. However, the user's guide could include more usage description rather than depending on the user to discover the "help" command within the program's execution. The download page contains a copy of the project for distribution, with the correct version number and release date. The distribution contains a ready-made "hale-aloha-cli-tiger.jar" file that can directly be used without a compile and build system. From the installer's perspective, the system was well-described and packaged. Again, based on the sample executions given above, the system can be improved in its date and error handling.
Developer's Perspective:
From the developer's perspective, the most useful starting point is the Developer's Guide. The guide fully covers the command-line based approach to automated testing and build, and provide a succinct set of guidelines with regard to new improvement development. It does not, however, enforce any project management process. It does, however, covers JavaDoc and Continuous Integration using Jenkins CI server.
Next, we take a look at the source code. From the source code, the JavaDoc documentation is generated, from which the system's design can be examined. The documentation is well written, and the system was designed well to support information hiding. However, we also notice that this particular design does not implement a CommandManager class, as suggested in the design specification.
With the Ant build system, it is easy to run the included tests and gain insight into its coverage. It should be noted here that two of the four commands are not tested by default: "energy-since" and "rank-towers". This is due to the fact that the developers named the JUnit tests incorrectly, and thus the tests are never invoked by junit.build.xml. This mistakes led to the poor total coverage of only 21%. Furthermore, the JUnit tests only exercises the "isValid()" command, while the main "run()" routine is never exercised in the tests. The testing part of this project can substantially be improved.
The source code is properly documented and easy to understand. However, it seems requirement capture was not rigorously exercised. The suggested modular design in the specification is not followed, and the commands are glued together in a very inefficient manner: requiring re-instantiation of every modules for each user input.
Next, we examine the team participation. Having prior knowledge that the team is following the "Issue-Driven Project Management" process, each members' contribution can be easily traced on the project's Issues page. There are two incomplete issues: "Issue 36: Improve JUnit Tests" and "Issue 19: Provide error checking in Command Processor." Thus, the developers are aware that their JUnit tests are incomprehensive. The issues page suggest approximately equal distribution of contributions.
The Continuous Integration page for this project also tells a part of the development story. The project is consistently worked on, and problems are promptly corrected. It appears there was a major issue with their build system from build #13 to #35, during which they struggled over two days trying to find and correct the cause. About 8 out of 10 commits were associated with an appropriate issue.
As a potential external developer, I can see many areas that can be improved upon. The build system seems to have been a major stumbling block, thus one should examine it and make it rock solid prior to starting the actual code development. Further, because some design specifications were not followed (i.e. modular design) new enhancements needs to change parts of the original code-base to be incorporated. Furthermore, with the inefficient instantiation, one might have to replace the Main glue codes completely.
It is understandable that "requirement capture" was not strictly stressed in the "Issue-Driven" development approach. But it surfaces eventually as a major issue in the validation process. Remember, design the system before you implement it!
From the installer's perspective, the project site has the basic description and user's guide needed to successfully install the system. However, the user's guide could include more usage description rather than depending on the user to discover the "help" command within the program's execution. The download page contains a copy of the project for distribution, with the correct version number and release date. The distribution contains a ready-made "hale-aloha-cli-tiger.jar" file that can directly be used without a compile and build system. From the installer's perspective, the system was well-described and packaged. Again, based on the sample executions given above, the system can be improved in its date and error handling.
Developer's Perspective:
From the developer's perspective, the most useful starting point is the Developer's Guide. The guide fully covers the command-line based approach to automated testing and build, and provide a succinct set of guidelines with regard to new improvement development. It does not, however, enforce any project management process. It does, however, covers JavaDoc and Continuous Integration using Jenkins CI server.
Next, we take a look at the source code. From the source code, the JavaDoc documentation is generated, from which the system's design can be examined. The documentation is well written, and the system was designed well to support information hiding. However, we also notice that this particular design does not implement a CommandManager class, as suggested in the design specification.
With the Ant build system, it is easy to run the included tests and gain insight into its coverage. It should be noted here that two of the four commands are not tested by default: "energy-since" and "rank-towers". This is due to the fact that the developers named the JUnit tests incorrectly, and thus the tests are never invoked by junit.build.xml. This mistakes led to the poor total coverage of only 21%. Furthermore, the JUnit tests only exercises the "isValid()" command, while the main "run()" routine is never exercised in the tests. The testing part of this project can substantially be improved.
The source code is properly documented and easy to understand. However, it seems requirement capture was not rigorously exercised. The suggested modular design in the specification is not followed, and the commands are glued together in a very inefficient manner: requiring re-instantiation of every modules for each user input.
Next, we examine the team participation. Having prior knowledge that the team is following the "Issue-Driven Project Management" process, each members' contribution can be easily traced on the project's Issues page. There are two incomplete issues: "Issue 36: Improve JUnit Tests" and "Issue 19: Provide error checking in Command Processor." Thus, the developers are aware that their JUnit tests are incomprehensive. The issues page suggest approximately equal distribution of contributions.
The Continuous Integration page for this project also tells a part of the development story. The project is consistently worked on, and problems are promptly corrected. It appears there was a major issue with their build system from build #13 to #35, during which they struggled over two days trying to find and correct the cause. About 8 out of 10 commits were associated with an appropriate issue.
As a potential external developer, I can see many areas that can be improved upon. The build system seems to have been a major stumbling block, thus one should examine it and make it rock solid prior to starting the actual code development. Further, because some design specifications were not followed (i.e. modular design) new enhancements needs to change parts of the original code-base to be incorporated. Furthermore, with the inefficient instantiation, one might have to replace the Main glue codes completely.
It is understandable that "requirement capture" was not strictly stressed in the "Issue-Driven" development approach. But it surfaces eventually as a major issue in the validation process. Remember, design the system before you implement it!
0 comments:
Post a Comment