Who Does the Testing

As I discussed in my last post there are many types of testing, but who should be doing the testing? This depends on what type of test is being conducted. Unit tests, regression tests and integration tests can be automated in many cases. They were probably written by the developers during development or by the QA department.

Automated tests are extremely good at demonstrating that the features and functionality work correctly. They can also be used to measure performance. By always running the same tests regressions can be identified quickly and we can be assured that existing features continue to work as new features are added.

However, there are many tests that simply cannot be automated, especially if your application contains a lot of UI elements. Sure there are programs that click buttons in a specific sequence and can "observe" the results. But, this isn't the same as a person interacting with the product. When a real person is clicking the buttons they an also evaluate how good the workflow is, how easy it is to move through the steps of the task and most importantly provide qualitative feedback.

These types of testers are a rare find. A good QA person needs to be someone who enjoys working on repetitive tasks or at the very least configuring several complex environments. The tester needs to be able to evaluate not only that the software is performing correctly but that it is doing the job in a usable way. Joel Spolsky has some advice on choosing testers.

If your program has a significant user interface you should include some Usability Testing into your schedule. These tests should be conducted in a semi-controlled environment with little direction. A user should be given a task to complete with the software and the session should be observed and/or recorded. This allows the developer(s) to see how easy their interface is to use. It can also identify places where additional error handling or explanation is necessary.

These kinds of tests should not be conducted by the engineer who built the interface. Let's face it, the interface that I build may make perfect sense to me because I know what it is supposed to do and how it does it. Using the form may not be as obvious to someone else working with the product. A sequence of actions to perform a task that is quite natural to me may be frustrating for many users. Jeff Atwood made a similar point in his article "Open Source Software, Self Service Software" when discussing self-service checkout.

There are certain rituals to using the self-service checkout machines. And we know that. We programmers fundamentally grok the hoops that the self-service checkout machines make customers jump through. They are, after all, devices designed by our fellow programmers. Every item has to be scanned, then carefully and individually placed in the bagging area which doubles as a scale to verify the item was moved there. One at time. In strict sequence. Repeated exactly the same every time. We live this system every day; it's completely natural for a programmer. But it isn't natural for average people. I've seen plenty of customers in front of me struggle with self-service checkout machines, puzzled by the workings of this mysterious device that seems so painfully obvious to a programmer. I get frustrated to the point that I almost want to rush over and help them myself. Which would defeat the purpose of a.. self-service device.

I related to this example right away. It seems very natural to me to do things step by step since that is the way we have to write programs. Most people don't break tasks down into these small chunks. A good tester can identify where these steps seem onerous which allows the developers to streamline the task.

Finally if you need some convincing on hiring testers take a look at Top Five (Wrong) Reasons You Don't Have Testers.

The Power of Testing

Testing is an often dreaded but very necessary part of the development process. It can be a tedious task but it can really pay off in the long run. I think this (de)Motivational poster sums it up nicely.

I realize that there isn't time to test everything. Like all programming tasks they need to be added into the schedule. Therefore, you need to choose the tests that best suit your situation. Here are a few examples.

• Unit Testing
• Regression Testing
• Integration Testing
• Usability Testing

Unit testing was brought to the forefront with the concept of Test-driven Development (TTD). TTD encourages developers to write a unit test first then create the method. This ensures that every method has a test written for it. If you are trying to apply this to an existing project it can be a very large task. For new projects there are many tools available which which help automate the process.

Creating unit tests for every function can be a time consuming process and needs to be balanced with generating the code that actually does the work. This can be further complicated since you need a positive and negative test in many cases. You need to ensure your function returns the correct results for valid input as well as proper handling of invalid input. Achieving the right amount of unit tests can be a somewhat daunting task.

Two types of tests which are almost always used are regression and integration tests. Regression testing ensures that changes to the code don't break any existing functionality. This is a step up from unit level tests in that it demonstrates that all of the units work together properly. Testing of new features is generally referred to as Integration Testing since it tests the addition of new features with the old.

When you are developing a program that will be used by people frequently Usability Testing is a valuable tool. Many people think that this can be a very time consuming and expensive type of testing to do. However, it doesn't have to be. You can conduct informal usability tests by simply grabbing someone in the hall and asking them to look at a form or web page that you just completed. If they can use the form then your on the right track.

Other simple usability tests can be conducted in a few hours using screen sharing software and a little help from your customers. This allows your developers to watch a user work with the program and identify good and bad aspects of the interface. I believe this kind of feedback is invaluable, it will lead to improvements and is well worth the time investment.

There are many other categories of tests that can be run and ultimately the level of tests you perform is going to depend on many factors. Although testing is very important delivering a product to market is just as important. Do enough testing to ensure that your product will work correctly and effectively then get it out to your best testers; your customers.

FAQs – January 2010

Client Communication Errors

Clients can get disconnected from Advantage in many ways. The network connection could fail, the client could loose power or the user could simply close the program abnormally. If the server detects that the client has been disconnected it will log a 7020 error.

However, there are some communication errors that can generate an error on the server. In rare cases a 9094 error, which will also generate a crash dump file, can be caused by a client communication failure. This should be resolved in version 9.10.0.20 which added a new error code 7213 which does not generate a crash dump file. This error is logged when the client gets a temporary or non-fatal communication error when sending a request to the server.

Two knowledge base articles that address this issue have been posted: 7213 Error Logged in the Advantage Error Log and Error 9094 Logged and Dump File Generated

"Empty" Date Fields in VFP Tables

When a date field is empty in a Visual FoxPro (VFP) table the Advantage ODBC Driver will return 12/30/1899 as the date. This is the default return value when an Xbase field contains an empty value.

You can avoid this situation in one of two ways. VFP allows fields to be created to support NULLs. When the field is created this way Advantage will return NULL if a value has not been defined.

Alternately you can check the field for an empty value by using the EMPTY() function. This is available as both an SQL scalar function as well as a filter expression. The EMPTY() function will return true if the field contains the Xbase empty value for the given field type.

SELECT * FROM MyTable WHERE EMPTY(MyDate) = TRUE

Can a Server Replicate to Itself?

Yes, an Advantage server with replication enabled can replicate to itself. You simply have to create another database on the server and configure it as the subscriber. This could be very useful in keeping an up to the minute backup of your data. Create the second database on another drive within the server so if the primary drive failed you would have a current backup immediately available.

You could also replicate to an external drive or to a Network Attached Storage (NAS) device for an additional measure of security.

Creating a Mini-Dump for a Specific Error Code

Advantage automatically creates a crash dump file whenever a 9000 class error occurs. These files assist the R&D team in determining what is going on with the server when the error occurred. Generally this allows us to quickly determine the problem and recommend solutions.

You can also create a mini-dump when you want more information about why a specific error code is being generated. This is done by adding some additional keys to the registry. Create registry entries named MINIDUMPERROR, MINIDUMPFILE, and MINIDUMPLINE in the ADS registry location of HKLM\SYSTEM\CurrentControlSet\Services\Advantage\Configuration.

To generate the log file just find the Error Number, File Name and Error Line you wish to create the dump for in the Advantage Error Log (ADS_ERR.adt). Then do whatever operation that caused the error again. Advantage should generate a file named adsdump*.gz on the root of the C drive, or the location configured in the error log path.

For more information about this process see this knowledge base article.

The Apple iPad

I am always amazed at how good a show Apple puts on. They do a spectacular job at releasing just enough information and rumor to whip everyone into a frenzy. Take the iPad announcement this week it has generated a avalanche of blog posts and tweets (twitters?) over the past two days. It ranked second on Twitter to Haiti on the 27th when it was announced. What would it be like to generate that kind of interest in your product?

So, I'll admit it I am interested in this new device. I have really enjoyed using my iPhone and I must admit the iPad looks like a really cool and fun device. Apple's multi-touch interface is the most intuitive interface that I have ever used. It is very easy to learn and it makes the device easy to use for just about everyone. It is so quick to learn how to navigate around and use the applications. I believe that this device will be very well received and Apple will sell millions of them.

The iPad is essentially a giant iPod Touch, which will be great for surfing the Web and watching movies but not so good at fitting in your pocket. The standard applications like calendar, contacts and photos have all been enhanced to make use of the larger screen size. It also adds the capability to read books. I think this is a great selling feature and it will appeal to a wide audience.

Since I already have an iPhone and a Kindle I don't see this as a necessary device for me. The iPhone is more portable, provides Internet access virtually everywhere and is my phone. I also think reading on an electronic ink screen instead of an LCD is much easier on the eyes. I really enjoy reading on my Kindle and I find it almost exactly the same as reading ink on paper.

If I didn't already have the other devices I would be much more interested in purchasing an iPad. I really like the instant on capability and it is large enough to comfortably surf the web. Books are also a great feature and I must admit the product video made the photo browser look really fun.

For all of the good/great things about the iPad I do see some drawbacks. It runs the iPhone OS which means no multi-tasking. On a device this big I would want to be able to have a few apps open at the same time. There are no cameras on the iPad, which I found surprising since this would be a perfect video chat device. It may be a bit awkward as a camera or camcorder though.

I found a couple of great articles on Gizmodo discussing the iPad. The best one "The iPad is the Gadget We Never Knew We Needed" has some great points and is well worth the read. For the downside to the iPad take a look at "8 Things That Suck About the iPad". Stephen Fry also had some great points in his article. I thought his comment to naysayers was quite good:

There are many issues you could have with the iPad. No multitasking, still no Flash. No camera, no GPS. They all fall away the minute you use it. I cannot emphasize enough this point: “Hold your judgment until you’ve spent five minutes with it”. No YouTube film, no promotional video, no keynote address, no list of features can even hint at the extraordinary feeling you get from actually using and interacting with one of these magical objects.

I would love to have a chance to use the iPad but I am sure it would make me really want to have one. I think this first generation will be good but will have its share of issues. However, I wouldn't turn one down if it was offered to me. I'm just not sure I'm ready to spend $500 on a device that has some glaring limitations. For now I would rather surf the web with my laptop at home or on my iPhone when I am mobile.

All that said I am sure Apple will sell millions of these and they are always very good at responding to their critics. I bet the next generation iPad will be a must have.

January Advantage News

The January edition of the Advantage Newsletter was sent out today. If you didn't get one you can view it here or you can sign-up and get it right when it is published. You can see a complete archive of the newsletter here.

Some of the exciting things announced in the newsletter include

• Advantage Database Server 10 | Pure & Simple webcast series – this three part series will introduce you to all the new features and improvements for the latest version of Advantage. This series begins in February and you can sign up here.
• The next Advantage Technical Summit will be held in Boise, Idaho on April 28th and 29th. This is a great opportunity to learn more about Advantage and network with other Advantage users. More information about this event will be sent out soon.

Keep watching for more event announcements.

My 2010 Reading List

As I mentioned in my first book review of the year I am going back to basics. I thought it was time to review some of the basic database theory along with solid design principals. With that in mind here are the titles that I have identified so far.

Database Stuff

• Database Design for Mere Mortals
• Data Modeling Essentials
• Database in Depth: Relational Theory for Practitioners
• SQL and Relational Theory: How to Write Accurate SQL Code
• The Art of SQL

User interface design and usability

• Emotional Design: Why We Love (or Hate) Everyday Things
• The Design of Future Things
• About Face 3: The Essentials of Interaction Design
• Rocket Surgery Made Easy: The Do-It-Yourself Guide to Finding and Fixing Usability Problems
• Writing Effective Use Cases

Miscellaneous

• Programming Pearls (2nd Edition)
• More Programming Pearls: Confessions of a Coder
• Coders at Work
• The Productive Programmer (Theory in Practice (O'Reilly))
• Peer Reviews in Software: A Practical Guide

A fairly ambitious list but I hope to get through them all this year. Stay tuned for book reviews as I get them read.

Book Review – Beginning Database Design

I wanted to take some time and go back to some database basics so I looked for some introductory books on database design. I decided on Beginning Database Design From Novice to Professional by Clare Churcher. This is a well written book with decent examples but a bit too introductory for a database programmer. I think it would be a very good book for someone who is looking to move from spreadsheets into databases though.

The book is relatively short at 240 pages. It contains twelve chapters along with a conclusion and index. It is truly aimed at the casual computer user who has a need to track data. Most of the examples pertain to education and research. Each of the examples are well thought out and clearly demonstrate the concepts she is discussing.

I thought her approach to introducing the material was well organized. She made some excellent points in her first chapter "What can go wrong", it demonstrated the importance of good design quickly. Chapter 2 is a brief introduction into the development process. I was happy to see some discussion of use cases in this chapter. I think this is a very important step in the design and development process.

Chapter 3 examines use cases more thoroughly showing how the process of working with the users and identifying how they will use the data is a fundamental requirement for designing a good database. Chapters 4 and 5 dive into creating a data model. This includes considerations of data types and logical grouping of data. These chapters also spend a lot of time discussing data relationships.

Chapters 6 discusses looking at data as classes. This provides a good way to demonstrate when it is appropriate to create subclasses that have some common data but also include specific information of their own. Chapters 7 through 9 discuss the relational database model. This is where all of the concepts introduced in the first half of the book come together. There is a lot of discussion about primary keys, normalization and constraints. Each of these concepts include simple and effective examples.

Chapter 10 is a very quick overview of SQL queries. It demonstrates how to create simple queries to get the data out of the database. If you have worked with SQL before you won't see anything new here. Chapter 11 discusses some user interface issues. I liked how she brought the UI design back to the use cases she discussed in the beginning of the book. All of the UI examples were done in MS Access and are very simple.

The book concludes with some discussion of object-oriented databases and using data within a spreadsheet. Probably very useful for a casual computer user but not something a programmer really needs. The conclusion does a good job of summarizing the material in the book as well as offering recommendations for more information.

The bottom line: this book is well written for its intended audience casual users to possibly IT professionals. As a database programmer you will want to invest in something more focused on a programmer's prospective.