Warning: Can't synchronize with repository "(default)" (/home/git/ome.git does not appear to be a Git repository.). Look in the Trac log for more information.
Notice: In order to edit this ticket you need to be either: a Product Owner, The owner or the reporter of the ticket, or, in case of a Task not yet assigned, a team_member"

Task #11269 (closed)

Opened 9 years ago

Closed 9 years ago

Deploy and evaluate MozTrap

Reported by: bpindelski Owned by: pwalczysko
Priority: minor Milestone: Testing and Docs
Component: QA Version: n.a.
Keywords: n.a. Cc: ux@…, cxallan, mtbcarrol
Resources: n.a. Referenced By: n.a.
References: n.a. Remaining Time: n.a.
Sprint: Testing and Docs (2)

Description

To allow for a streamlined experience during scenario testing, MozTrap? has been considered a valuable addition to the set of tools used by the team.

It has been deployed (internal access only) at https://moztrap.openmicroscopy.org. Setup documentation left in OME google docs. Petr has been assigned the role of Test Manager, so the next steps will be to convert a scenario and have a test run of the system with the team involved.

Attachments (12)

Screen Shot 2013-07-19 at 17.09.29.png (137.5 KB) - added by pwalczysko 9 years ago.
Screen Shot 2013-07-19 at 17.09.49.png (47.5 KB) - added by pwalczysko 9 years ago.
Results per case.png (275.6 KB) - added by pwalczysko 9 years ago.
Status of test result.png (202.6 KB) - added by pwalczysko 9 years ago.
Test during run.png (135.2 KB) - added by pwalczysko 9 years ago.
Test during run - comments & history.png (146.5 KB) - added by pwalczysko 9 years ago.
assignment of test cases to users.png (114.9 KB) - added by pwalczysko 9 years ago.
Creation of test cases - free copy and paste.png (141.7 KB) - added by pwalczysko 9 years ago.
Execution list with all executions.png (229.1 KB) - added by pwalczysko 9 years ago.
Execution prepared to test - estimated duration of the test included.png (61.6 KB) - added by pwalczysko 9 years ago.
REsults view with more results.png (176.0 KB) - added by pwalczysko 9 years ago.
check the execution was done.png (28.4 KB) - added by pwalczysko 9 years ago.

Download all attachments as: .zip

Change History (23)

comment:1 Changed 9 years ago by bpindelski

Assigning to pwalczysko for general demo preparation - i.e. moving of a selected scenario into moztrap. Next step will be a demo session with the team.

comment:2 Changed 9 years ago by bpindelski

  • Owner changed from bpindelski to pwalczysko

comment:3 Changed 9 years ago by pwalczysko

@bpindelski: On Friday at 5 pm tried to login with my LDAP credentials. This went fine. Was able to create test suites, test cases, test runs.
But for the Environments, I have massive problems. Don't seem to be able to create a new Environment profile and add Environments to it.
More exactly, can create an Environment Profile (such as OS or Browser) and edit it. But cannot add a new Environment (such as Linux or Firefox).
When I edit the Environment Profile and go "Add Environment", I am allowed to add for example Firefox and anything I like into the line.
Then I click "Done Editing". Then I edit the Environment Profile again, but no Firefox is shown - looks like my previous editiing was ignored.
Is it a permissions issue ?
I cannot run tests now, because I cannot add the Environment to the Product Version. When I try to add these, it goes as above - apparently allowing me this, then not performing the action.
I cannot run the test without an environment.

See screenshots.

Last edited 9 years ago by pwalczysko (previous) (diff)

Changed 9 years ago by pwalczysko

Changed 9 years ago by pwalczysko

comment:4 Changed 9 years ago by bpindelski

All the limitations have been discovered and worked around during sessions spent with Petr. A test of moztrap among the team is planned for 31/07/2013. Feedback will be collected and reacted upon.

comment:5 Changed 9 years ago by bpindelski

Smoke test results and feedback have been collected in a g.doc (MozTrap? testing). For reference, the development tracking page for moztrap is https://www.pivotaltracker.com/s/projects/280483.

comment:6 Changed 9 years ago by bpindelski

The discussion around the testing results has been captured in a g.doc (MozTrap? feedback). General next action points are:

  • Take sphinx docs from moztrap and start modifying it for ourselves
  • Work on test assignments to users
  • Add a g.doc spreadsheet-style view of results
  • Try out API calls with a "automated-test" user and moztrap-connect (https://github.com/camd/moztrap-connect/)

comment:7 Changed 9 years ago by bpindelski

  • Sprint changed from Testing and Docs (1) to Testing and Docs (2)

Referencing ticket #11258 has changed sprint.

comment:8 Changed 9 years ago by pwalczysko

@jburel @jamoore @bpindelski @mtbcarrol
After testing of MozTrap? we are having a look at Tarantula.
Considering the points raised during the test of MozTrap? and asking : Are these points solved by Tarantula ? The answer is yes. See also screenshots.
Went to http://www.testiatarantula.com/features/demo/ - username user, pwd user. (Also this link is helpful to get started http://www.testiatarantula.com/features/getting-started-with-tarantula/)
Went through the workflow of creating an Annotate scenario (Test Case) for Insight and Web.
Points we were looking for (sorted according to importance):

  1. It is not possible to do comments on every step in MozTrap? as you are testing - this is sorted out by Tarantula - you can do comment at every step, also you can see where you are in the test, what was done, what not. Also, in Tarantula you can fail a step and still be allowed to test the scenario to the end (not possible in MozTrap?.
  2. Overview of the results is poor in MozTrap?, need a convenience of the Googledoc - we have this solved in Tarantula using "Case execution list" under "Report" tab - see the results as per Test Case or as per "Execution" (=Test Run in MozTrap?). Tarantula gives us a table with obvious, immediate info about who TESTED it, lists ALL the comments on the first page (no need to click through the links to get to see the comments). If you need to see single steps in the Test Case (which steps were were failed, which were passed exactly, you can see it under Test tab - when you open the correct execution, you will see a history of the test at a glance as well. Executions tab shows additionally the progress of the test cases, and again, who was assigned, who executed at first glance, and whether the test was run or not yet. Also you can see how long did it take for the tester to run the test.
  3. Assign tests to people - sorted by Tarantula, in every Execution every Test Case is compulsorily assigned to a tester - does not matter if the tester does not have the time to execute it and asks a colleague to help him out and execute the test for him - Tarantula will remember both persons, and hightlight the more important one (= who actually tested) in the results / overview tables

Furthermore, the pluses for Tarantula are also the ease of writing scenarios, I have unlimited possibility to copy steps from any scenario into any other scenario (no cloning like in MozTrap?, no restrictions). I like this.

Drawbacks of Tarantula:

  1. Screenshots - not really obvious at this time how this should work, markup impossible according to Blazej. Possibly we could go away without screenshots ? Up to the testers, I don't need them.
  2. (A Minor one) - not seeing now how to link a ticket from Trac to the test comment like it was possible with MozTrap? - on the other hand Tarantula directly supports connection to Bugzilla and jira - Jmarie is looking into the possibility to widen this to Trac -> this would give Tarantula immediate plus against MozTrap? as well.
  3. LDAP not possible - absolutely no problem for me - as I understand it, the fear here is that we will have to create all the users manually -> more than happy to do this myself - creation of even 30 users (pretty unrealistic number, actually we talk about ~10) is just nothing in comparison to creating and maintaining ~100 scenarios. I would probably need to be an admin to do this, but why not ?

In general, pretty happy with Tarantula. Seems that there are just pluses, except the Screenshots problem. Certainly worth a serious try.

Changed 9 years ago by pwalczysko

Changed 9 years ago by pwalczysko

Changed 9 years ago by pwalczysko

Changed 9 years ago by pwalczysko

Changed 9 years ago by pwalczysko

Changed 9 years ago by pwalczysko

Changed 9 years ago by pwalczysko

Changed 9 years ago by pwalczysko

Changed 9 years ago by pwalczysko

comment:9 Changed 9 years ago by pwalczysko

  • Cc ux@… cxallan mtbcarrol added

comment:10 Changed 9 years ago by jburel

Petr: you can add the screenshots to the case itself using the "add attachments" option.

comment:11 Changed 9 years ago by jamoore

  • Resolution set to fixed
  • Status changed from new to closed

Closing since we are likely to evaluate tarantula after Petr's return. The investigation here is all still quite value and if this needs to be re-opened, please do so.

Note: See TracTickets for help on using tickets. You may also have a look at Agilo extensions to the ticket.

1.3.13-PRO © 2008-2011 Agilo Software all rights reserved (this page was served in: 0.69823 sec.)

We're Hiring!