Sunday, September 27, 2009

One Mission - Many Approaches: BWT 9 Experience Report


Date: 26th September 2009
Time 3pm - 5pm IST
Product Tested: SplashUp

Mission: To find Functional bugs in the Splashup application

Last week, we had tested the same application with testers choosing different quality criteria.
This week, we had to test the same application with one quality criteria - "Functionality" as the base.

Testers: Ajay Balamurugadas, Amit Kulkarni, Dhanasekar Subramaniam, Gunjan Sethi, Karan Indra, Parimala Shankaraiah, Poulami Ghosh, Rajesh Iyer and Suja C S.

This was the first session where I was not moderating and only testing the product.
I was happy that I could dedicate more time for testing.

We tested from 3pm to 4pm and started the discussion session at 4pm sharp.

Poulami started off the discussion. This being her first experience with BWT, used Exploratory Testing Approach to guide her. She wanted to get a feel of the product before she could concentrate on issues in the application.

She found the "Layers" feature interesting enough to continue her focused testing on the Layers and Filters feature.Happy with her first BWT experience, she promised to attend more sessions before passing any feedback to the team.

Poulami found the application very user-friendly and found the Auto-Crop feature not working.

Rajesh was next to describe his experiences. He was interested more in the Sign Up feature of the product. Having created an email address with username of 132 characters length, he was unable to login. Though the email was created successfully, an error message greeted him on Login.

Me and Rajesh had a discussion about an error message popping on the screen if Webcam was not connected. We were not sure if Flash generated the error or the SplashUp application generated this error. While I felt that the error was application specific, Rajesh was of the opinion that it was similar to Flash Generic messages.

I was happy that Rajesh enjoyed testing the application. He also felt that this was a good application to test.

Once Rajesh was done with his description, Amit took over. Amit was frustrated with the application being non user friendly. Absence of help files and lack of support to other image formats posed a serious question regarding the scope of the application.

One of the highlights of Amit's description was the bug he discovered. Moving the error message out of visible window area made it disappear.
He felt that such bugs were common in similar applications and make him wonder if the application is really tested before releasing.

Someone had to cool Amit's frustration on the product and Dhanasekar took centre stage. Like Poulami, he too was a first timer to BWT. He had no experience of testing any imaging software and hence concentrated on the different file types for the application.

One of the bugs found by Dhanasekar was the "Improper handling of unsupported file formats".

This made me wonder how different people look at the same application in different ways and How thought process of each individual under the same circumstances varied.

The only concern he expressed was the lack of prior knowledge of the product being tested. BWT's purpose of letting testers to test with less information about the product would be defeated. The thrill of testing an application when one does not know anything about the application is different from testing a known application is different.

There is less chance of getting biased if one does not know much information about an application. Amit also was of the opinion that exploring a product without much information is good as testers get to learn a lot of new things.

What followed next interested me. Suja's description of her testing approach. After the initial "Get to know the product" session, Suja divided her tests into "Happy Testing" and "Negative Cases".

I feel this is a very narrow way of modelling the application. It was good to see other testers actively participating in the discussion. Even Suja wanted the application to have more documentation to help the user. The experience with BWT was good and she was happy.

Gunjan was next and her previous experience in testing imaging software helped her. using an Exploratory approach, she went on different tours of the product. She found some bugs with the Zoom and Filters feature. Her logical approach to testing the application was a different experience when compared to the last BWT session she attended.

Her only concern was that it took some time to know some features.

Next was my turn. Only testing and no moderating was in itself a different experience for me. This application had lots of bugs and if one is Bug-hungry, I'd recommend this application.

One of the strange bugs I discovered was to make the Menu bar disappear.I also learnt a lot of different bugs.

The purpose of BWT is achieved if a tester goes back with some learning. :)

Amit asked a very important question:
How many of you tried using the application only with their keyboard?
I replied in the negative as if it failed, that would be an usability issue and the mission was to find functionality issues.

Karan's summary was rocking. He had typed everything in a notepad and just pasted everything at once on his turn.
Following an Exploratory approach to some extent, he felt the application was not user friendly. He was confident that with time, this application could be developed into a full-fledged application.

Parimala - the moderator for the session was the last one to present.
Lack of dedicated time for testing was her main concern.
A new software for her, being a curious tester, she explored and learnt most of it quickly. She tested the Tools section of the application till time permitted.

Overall the session was good coupled with strange bugs an discussions about them.
The only concern was: It was fast and discussions were not full-fledged.

We will improve on this next time.
Thanks to all the testers, I learnt some new bugs.
Interested to join us in next session? Email to weekendtesting@gmail.com

See you all in BWT 10.
Till then, ENJOY TESTING :)

Update: Please find the Test Report shared at Scribd.

Leia Mais…

Friday, September 25, 2009

Weekend Testing Session No. 9

Friends,

Weekend is nearing and so is our testing session. Please confirm your participation for the "BWT Session No.9"

Date: Saturday 26th September 2009
Time: 3pm – 5pm IST

Please be online on Gmail (visible mode) by 2.30pm IST.
You’d be provided download details.

Testing session: 3pm – 4pm IST
Discussion Time: 4pm – 5pm IST

Please send an email to weekendtesting@gmail.com with the subject “BWT 9 Confirmed Participant”.

We’ll include you for the session once we receive an email.

For more details, contact weekendtesting@gmail.com

Leia Mais…

Monday, September 21, 2009

Challenge of dimensions of quality : BWT 8 : Experience Report

20th September 2009, 9pm -11pm IST would be etched in the minds of six testers who got together online to test the ‘Areca Backup’ application. This session marked the eighth session of BWT. It was exactly two months since the concept of ‘Bangalore Weekend Testers’ originated.

List of Participants: Ajay Balamurugadas, Bhargavi, Gunjan, Parimala Shankaraiah, Sudhakar, Vasupratha

Application: Areca Backup

In their own words…
Areca-Backup is a file backup software that supports incremental, image and delta backup on local drives or FTP servers. Areca-Backup also allows you to browse your backups and navigate among different version of the files contained in your archives.”

All the testers were geared up for the testing session.
The application had been downloaded.

What next?
What about the MISSION?

The mission for this session was as special as the session.

Following mission was given to the testers:
Mission -: You have to choose one of the quality criteria out of the following –
Installability / Usability / Performance / Reliability / Compatibility /Testability.
Choose one quality criteria and stick to that quality criteria for the entire session to test the ‘Areca Backup’ application.


Special THANKS to Pradeep who suggested this mission.

Each tester was very enthusiastic on hearing the mission and started their journey of exploring the product in order to find valuable information.
The testing session started at 09.03pm and lasted till 10.03pm IST.
We had the Discussion session soon after the testing session.

Each tester was supposed to reveal the mission they chose, the approach followed during the testing session. The tester had to highlight any challenges they faced and any specific learning from this session. The individual tester’s experience of the BWT 8 session was the icing on the cake :)

Sudhakar, the first tester to send in his Test Report, started off the discussion session.
He was very clear in his mission:
“Find issues” and he chose Usability as the Quality Criteria.
One interesting thing about his whole approach to test this application was his expectation: “The application should guide the user”

With focus on data validations, Sudhakar was frustrated at the not-so-good validation implemented.
One major challenge apart from the poor validation was the time taken to understand the application. Lack of understanding the product fully prevented him from exploring the product to a greater degree. Finally, Sudhakar felt other than lack of time, the overall experience of participating in this session was good.

We moved on to Vasupratha’s experience.
Vasupratha echoed Sudhakar’s concern about the lack of time. Usability was the quality criteria once again. Vasupratha felt that additional time for testing would have helped in better exploration of the product.

Next turn was Parimala’s.
A different Quality Criteria: Installation was chosen.
The mission set by her was straight forward:
“To test the installability of Areca 7.1.5”
Following an Exploratory approach, Parimala gave a lot of valuable information in terms of bugs. As the number of installation steps was minimal, Parimala did not face a lot of challenges.
At the same time a particular intermittent bug was playing hide and seek with her. :)

Parimala learnt some new scenarios to test once she took up Installation as the criteria. The new learning (New Scenarios) helped her do a round of Scenario Testing.
With this being a good experience, she wanted to do some functional testing in the near future. :)

Gunjan was ready to share her experiences.
Her mission was decided more because of the circumstances than her choice.
Usability was her first choice. The application when launched was giving an error about a missing .dll file. So, Gunjan shifted her focus from Usability to Installability as she had to un-install and re-install the application.

With an exploratory approach to her rescue, Gunjan delved deep into issues in installation and un-installation. Some interesting issues greeted Gunjan even though System restore was also tried to get the application working.
The help file was one of the sources of information from which she tried out the scenarios. Her biggest learning was to ensure system is in correct condition before testing any application.
Gunjan being a first timer to BWT enjoyed herself and found it interesting to think “OUT OF THE BOX”. This was the first time she tested any software out of her office work.

Now, it was the turn of Bhargavi.
Bhargavi’s mission focused on finding problems with Performance as the Quality Criteria.
Following an Exploratory approach, Bhargavi could face many challenges easily.
The major challenge was the difficulty in understanding the features and knowing where to start and how to start modeling the application.
Some bugs pertaining to other quality criteria slowed down Bhargavi’s progress.

She had her share of learning too. As she took the “Performance” quality criteria which she hadn’t tested before, she learnt new ideas to test. This boosted her confidence. Bhargavi enjoyed testing the product with a different perspective – Focus on only one quality criteria.
Her tests forced 100% CPU usage as well as low disk space.

The mission taught Bhargavi to concentrate on particular quality criteria who habitually concentrated on all quality criteria. Parimala added a point as to how testers find interesting and intriguing issues when focus is on a small part of the application.

Finally, it was my turn.
I chose “Performance” as the quality criteria for the simple reason: Never tested for Performance alone before. I too followed an Exploratory approach with my toolkit which consisted of Process Explorer, Windows Task Manager, MS Excel, Notepad, WebEx Recorder and Date and Time Properties Window.

The biggest challenge for me was to learn the product quickly.
Help file helped me to some extent.
Once I understood how to backup, I started with 23GB folder and that was my biggest mistake of the day. :(

Expecting backup software to handle 23GB of data and backup within 15 minutes was very foolish on part of me. Thereby, I spent fifteen minutes watching the progress bar of the backup process.

On trying with a 4MB file, backup process completed within a matter of few seconds.
I glanced through the report which was generated after backup. A bug in the report took away my precious ten minutes.

Biggest learning I had out of this exercise was to prepare test data while the system was being modeled. Also having unrealistic goals(Read 23GB file) does not help the cause.

Later, I tried with 30MB, 60MB and 90MB folder to monitor the performance. But it was almost the end of testing session.
Bharath highlighted the value www.testersdesk.com added in testing the Performance criteria.
Experience was good as it marked the successful completion of two months of Weekend Testing.

Every BWT session gave me a different experience.
Right from BWT 1 where I and Parimala tested www.vischeck.com to BWT 8, every experience is a unique learning and thought provoking experience.

I’d like to thank all the testers, Pradeep and the BWT members: Manoj, Parimala and Sharath for their continuous support and hard work.

Looking forward to BWT 9: A new product, new testers and a new experience.
See you there. :)

Leia Mais…

Saturday, September 19, 2009

A five minute conversation with a programmer

This is a post highlighting the conversation I had with one of my programmers.


Programmer (P): Hi Ajay! I need your help in reproducing the defect #abcdef

Defect #abcdef:
Step 1: ...
Step 2: Enter a value 50 in the text field.
Step3: ...

Ajay (A): Sure, How can I help you?

P: I'm unable to reproduce the defect.

A: Which OS are you trying it on? I had logged it on Win 2003.
P: Yes, I know that. It would be easier if I could reproduce that on Win XP.
A: (he he smiles) OK, I'll reproduce the defect first on Win 2003 and then we could try on Win XP also.
P: OK, great.

A: Step1, Step2, Step3 and here it is, REPRODUCIBLE!!!
P: Ok, let me try it. Step1, Step2 and Step3 and ??? Where's the defect?
A: Oh!!! Let me try again. Step1, Step2 and Step3 and again REPRODUCIBLE!!!
P: (Smiles) Step1, Step2 and Step3 : NOT REPRODUCIBLE

Silence for few seconds.

A: (thinking what could be different) Hmmm, maybe the speed with which I execute the steps is different from your speed of execution.
P: Maybe. Very little chance of that happening.
A: OK, let me try consecutive times. Step1, 2, 3: REPRODUCIBLE. Step1, 2, 3: REPRODUCIBLE.
P: Oh!!! How do YOU reproduce that Man!!! See, Step1, 2, 3: NOT REPRODUCIBLE.

P: Maybe you are pressing 'Enter' after entering the number.
A: Hmmm, I'm not pressing Enter key.
P: Then maybe single click causes this problem. Or maybe double click to select the field before entering the number.

A: (thinking WOW, so many factors!!! Let him go on)
P: Or this might happen if you select the text by 'Click and Drag' using mouse
P: OK, I'll look into this issue. Thanks for your time.
---------- END OF CONVERSATION ---------

This particular conversation refreshed my memory of how many different factors affect a single entry in a text field.

> Operating System
> Response time
> If the focus is on the field or not
> After entering the number, did the user press Enter or Tab?
> Did the user double click on the field to select the default text?
> Did the user delete the text before typing in the new text?
> Did he press 'Delete' or 'Backspace'

I'm sure there are many more factors related to the single entry in the text field.

The point I want to highlight here is "How useful is it to have a conversation with a programmer about the product?"

In my case, it was useful. Have you experienced such an interaction?
Do let me know.

Please feel free to share such experiences(Good & Bad).

Leia Mais…

Friday, September 18, 2009

Weekend Testing Session No.8

Friends,

Weekend is nearing and so is our testing session.

BWT Session No.8

Date: Sunday 20th Sep 2009
Time: 9pm – 11pm IST

Please be online on Gmail (visible mode) by 8.30pm IST. You’d be provided download details.

Testing session: 9pm – 10pm IST
Discussion Time: 10pm – 11pm IST

Please send an email to weekendtesting@gmail.com with the subject
“BWT 8 Confirmed Participant”.

We’ll include you for the session once we receive an email.

For more details, contact weekendtesting@gmail.com

Leia Mais…

Friday, September 11, 2009

Weekend Testing Session No. 7

Friends,

Weekend is nearing and so is our testing session.

BWT Session No.7

Date: Sunday 13th Sep 2009
Time: 5pm – 7pm IST

Please be online on Gmail (visible mode) by 4.30pm IST. You’d be provided download details.

Testing session: 5 – 6pm IST
Discussion Time: 6 – 7pm IST

Please send an email to weekendtesting@gmail.com with the subject
“BWT 7 Confirmed Participant”.

We’ll include you for the session once we receive an email.

For more details, contact weekendtesting@gmail.com

Leia Mais…

Monday, September 7, 2009

A Silent post dedicated to all the Silent Readers

This is a Silent Post to all the Silent Readers of my Blog.

A silent post which is not supposed to say anything but just the word "Thanks"

Leia Mais…

Sunday, September 6, 2009

Weekend Testing Session 6 - TuxPaint



How many times have you enjoyed a software while testing it?

One such occasion was in the Bangalore Weekend Testers Session held on Saturday, the 5th of September 2009 3pm - 5pm IST.

Twelve testers agreed to test TuxPaint, a free, award-winning drawing program for children ages 3 to 12. It combines an easy-to-use interface, fun sound effects, and an encouraging cartoon mascot who guides children as they use the program.

WOW!!! The kid in every tester came out and literally played with the software such that 60 bugs surfaced which were carefully noted down by the tester at work.

Amazing to know if you enjoy doing something, you can excel at it too!!!

Please do treat yourself with this software available at www.tuxpaint.org.
If you want to know the issues beforehand, check out the list of bugs found by us here.

Thanks to Bill Kendrick who permitted us to test and publish the report.

I'm sorry if you missed this week's session too but you have a chance next week.
If you want to register for next week, email to weekendtesting@gmail.com and watch out this space.

Leia Mais…

Thursday, September 3, 2009

Weekend Testing Session 6

Friends,

Next weekend is nearing and so is our testing session.

BWT Session No.6

Date: Saturday 5th Sep 2009
Time: 3pm – 5pm IST

Please be online on Gmail (visible mode) by 2.30pm IST. You’d be provided download details.

Testing session: 3 – 4pm IST
Discussion Time: 4 – 5pm IST

Please send an email to weekendtesting@gmail.com with the subject
BWT6 Confirmed Participant”.

We’ll include you for the session once we receive an email.

For more details, contact weekendtesting@gmail.com

Leia Mais…