Dies ist ein Archiv des alten Softwaretechnik Lehrstuhls der Universität des Saarlandes. Es ist nicht länger aktuell.


Testing and the Cloud
Seminar - Winter Semester 2015/2016

Lehrstuhl für Softwaretechnik (Prof. Zeller)
Universität des Saarlandes – Informatik
Informatik Campus des Saarlandes
Campus E9 1 (CISPA)
66123 Saarbrücken
E-mail: zeller @ cs.uni-saarland.de
Telefon: +49 681 302-70970

Deutschsprachige Startseite Page d'acceuil en français English home page

This seminar will explore the state-of-the-art in Testing AND the Cloud.
This includes existing methods that leverage the Cloud for improving some aspects of Software Testing (a.k.a., Testing IN the Cloud) as well as work on Software Testing of Cloud-based applications (a.k.a., Testing THE Cloud) and work on Testing of novel non-functional properties of Cloud-based applications (a.k.a., Testing Elasticity).

We will have weekly meetings. Each week we will discuss one research paper.

The kickoff-meeting will be on Thursday, 29.10 22.10, from 11.15 till 12.00 in room E1 1 1.06

Meetings will be on Thursdays from 11.00 till 12.00 in room E1 1 1.06


  • 11.02.2016 - There will be a double session, starting at 10.30
  • 04.02.2016 - No seminar this week. The chair of SE is in out for a retreat.
  • 28.01.2016 - No seminar this week. Alessio is sick.
  • 21.01.2016 - The poster session of the exam will not be done. Most likely, there will be a closing session with open-discussion instead. To be confirmed.
  • 10.12.2015 - No seminar this week due to our migration to the new CISPA building (E9 / 1). Room to be assigned.
  • 29.10.2015 - Kick-off Meeting [PDF]. Paper assignment and presentation date.
  • 29.10.2015 - The official list of papers is out.
  • 20.10.2015 - Open the waiting list for the seminar. (See Registration)
  • 19.10.2015 - The date of Kickoff meeting changed to Thursday, 29.10 (one week later than the original schedule).
  • 19.10.2015 - The registration for the Seminar is closed. We have reached the maximum number of participants for the seminar, and we do not accept any other.
  • 19.10.2015 - The meeting room and the Seminar schedule are confirmed.
  • 24.09.2015 - The meeting room and the Seminar schedule are tentative. We are checking the room availability.

Assignment of topic to students

Date Student Topic Description
05-11-2015 Kai Testing IN the cloud 1 Improve the efficiency of the parallel execution
12-11-2015 Sebastian Testing IN the cloud 2 Use the cloud for transparent execution of test cases
19-11-2015 Joris Testing IN the cloud 3 Using virtualization to improve the efficiency of system testing
26-11-2015 Valeh Testing WITH the cloud 1 Load testing and Automatic Saturation Detection
03-12-2015 Alexandria Testing WITH the cloud 2 Cloud 4 Security: Penetration Testing
10-12-2015 No Seminar
17-12-2015 Subhashini Testing WITH the cloud 3 Testing Service Oriented Architectures
07-01-2016 Karthik Testing THE the cloud 1 Functional Testing at IaaS level
14-01-2016 Emamurho Testing THE the cloud 2 Non-functional Testing at IaaS level
21-01-2016 Nour Testing THE the cloud 3 Functional Testing at PaaS level
28-01-2016 No Seminar
04-02-2016 No Seminar
11-02-2016 Mohamed Testing Elasticity 2 New security exploits
11-02-2016 Ezekiel Testing Elasticity 1 Iterative Generation of Test Suites for Testing Elasticity

List of papers organized by Area and Topic

The papers are available only if are inside the faculty network!

Background reading

  • [PDF] Robert B. Bohn, John Messina, Fang Liu, Jin Tong, and Jian Mao, NIST Cloud Computing Reference Architecture, IEEE World Congress on Services, 2011
  • [PDF] Steffen Kachele, Franz J. Hauck, Christian Spann, and Jorg Domaschka, Beyond IaaS and PaaS: An Extended Cloud Taxonomy for Computation, Storage and Networking International Conference on Utility and Cloud Computing, 2013
  • [PDF] Yucong Duan, Guohua Fu, Nianjun Zhou, Xiaobing Sun, Nanjangud C. Narendra, and Bo Hu, Everything as a Service(XaaS) on the Cloud: Origins, Current and Future Trends, IEEE International Conference on Cloud Computing, 2015
  • [PDF] Sadeka Islam, Alan Fekete, Kevin Lee, and Anna Liu, How A Consumer Can Measure Elasticity for Cloud Platforms, International Conference on Performance Engineering, 2012
  • [PDF] Nikolas Roman Herbst, Samuel Kounev, and Ralf Reussner, Elasticity in Cloud Computing: What It Is, and What It Is Not, International Conference on Autonomic Computing, 2013
  • [PDF] Guilherme Galante and Luis Carlos E. de Bona, A Survey on Cloud Computing Elasticity, International Conference on Utility and Cloud Computing, 2012
  • [PDF] Yike Guo, Moustafa Ghanem, and Rui Han, Does the Cloud Need New Algorithms? An Introduction to Elastic Algorithms, International Conference on Cloud Computing Technology and Science, 2012
  • [PDF] Alessio Gambi, Waldemar Hummer, Hong-Linh Truong, and Schahram Dustdar, Testing Elastic Computing Systems, IEEE Internet Computing, Volume 17 Issue 6, Date Nov.-Dec., 2013
  • [PDF] Philip Robinson and Carmelo Ragusa, Taxonomy and Requirements Rationalization for Infrastructure in Cloud-based Software Testing, International Conference on Coud Computing Technology and Science, 2011
  • [PDF] Xiaoying Bai, Muyang Li∗, Bin Chen, Wei-Tek Tsai, and Jerry Gao, Cloud Testing Tools, IEEE International Symposium on Service Oriented System Engineering, 2011

Area 1: Testing IN the Cloud

Use of the cloud or virtualization technologies to improve one or more aspects of the execution of test cases.
With cloud, everything might become a service. Testing is no different and can be provided as a remote service

Using the cloud to improve the efficiency of the testing process

  • [PDF] Stefan Bucur, Vlad Ureche, Cristian Zamfir, and George Candea, Parallel Symbolic Execution for Automated Real-World Software Testing, EuroSys, 2011

Using the cloud for transparent execution of test cases

  • [PDF] Philipp Leitner, Zabolotnyi Rostyslav, Alessio Gambi, and Schahram Dustdar, A Framework and Middleware for Application-Level Cloud Bursting on Top of Infrastructure-as-a-Service Clouds, International Conference on Utility and Cloud Computing, 2013

Using virtualization to improve the efficiency of system testing

  • [PDF] Alessio Gambi and Schahram Dustdar, A Technique for Improving Cloud-based Continuous Integration Environments, submitted to the International Conference on Automated Software Engineering
  • [PDF] Alessio Gambi, Zabolotnyi Rostyslav and Schahram Dustdar Poster: Improving Cloud-based Continuous Integration Environments IEEE International Conference on Software Engineering, 2015

Area 2: Testing WITH the Cloud

Use of the cloud or virtualization technologies to improve one or more aspects of the execution of test harnesses to generate load

Load testing and Automatic Saturation Detection

  • [PDF] Alain Tchana, Bruno Dillenseger, Noel De Palma, Xavier Etchevers, Jean-Marc Vincent, Nabila Salmi, and Ahmed Harbaoui, A Self-Scalable and Auto-Regulated Request Injection Benchmarking Tool for Automatic Saturation Detection, IEEE Transactions on Cloud Computing, VOL. 2, NO. 3, July-Sept., 2014

Penetration Testing

  • [PDF] Richard Li, Dallin Abendroth, Xing Lin, Yuankai Guo, Hyun-wook Baek, Eric Eide, Robert Ricci, and Jacobus Van der Merwe, POTASSIUM: Penetration Testing as a Service, Symposium on Cloud Computing, 2015

Testing SOA

  • [PDF] Steffen Herbold, Alberto De Francesco, Jens Grabowski, Patrick Harms, Lom M. Hillah, Fabrice Kordon, Ariele-Paolo Maesano, Libero Maesano, Claudia Di Napoli, Fabio De Rosa, Martin A. Schneider, Nicola Tonellotto, Marc-Florian Wendland, Pierre-Henri Wuillemin, The MIDAS Cloud Platform for Testing SOA Applications, International Conference on Software Testing, Verification and Validation, 2015

Area 3: Testing THE Cloud

Approached, methods, techniques and tools to test cloud-based applications or applications that leverage the cloud.

IaaS - Functional Testing

  • [PDF] Waldemar Hummer, Florian Rosenberg, Fabio Oliveira, and Tamar Eilam, Testing Idempotence for Infrastructure as Code, Middleware, 2013

IaaS - Non-functional Testing

  • [PDF] Deepal Jayasinghe, Galen Swint, Simon Malkowski, Jack Li, Qingyang Wang, Junhee Park, and Calton Pu, Expertus: A Generator Approach to Automate Performance Testing in IaaS Clouds, International Conference on Cloud Computing, 2012

PaaS - Functional Testing

  • [PDF] Stefan Bucur, Johannes Kinder, and George Candea, Making Automated Testing of Cloud Applications an Integral Component of PaaS, ACM Asia-Pacific Workshop on Systems, 2013

Area 4: Testing Elasticity

Approached and techniques to test (or at least understand) novel aspects of cloud-based applications, such as elasticity.

Iterative Testing of Elasticity

  • [PDF] Alessio Gambi, Antonio Filieri and Schahram Dustdar, Iterative test suites refinement for elastic computing systems, Joint Meeting on Foundations of Software Engineering, 2013

New security exploits

  • [PDF] Joseph Idziorek and Mark Tannian, Exploiting Cloud Utility Models for Profit and Ruin, IEEE International Conference on Cloud Computing, 2011


  • [PDF] Kirill Bogdanov, Miguel Peon-Quiros, Gerald Q. Maguire Jr., and Dejan Kostic, The Nearest Replica Can Be Farther Than You Think, IEEE International Symposium on Service Oriented System Engineering, 2015


We have reached the maximum number of participants for the seminar, and we do not accept any other. The registration is closed.

Since we received several requests, we decided to open a waiting list for the seminar. To register in the waiting list send an email to Dr. Alessio Gambi. If registered students drop off the class, we will contact the members of the waiting list. As always, we adopt a first-come-first-served policy.

Passing Requirement

What should you do in order to gain credit points for this course?

  • Submit all Summaries on time
  • Attend the weekly meetings. Up to 2 unexcused absences accepted. Please inform us if you have a valid reason to miss the discussion
  • Present at least one of the papers before the end of semester for 15-20 minutes
  • Submit annotated slides of your talk post presentation
  • Additional criteria presented in class

Grading Scheme

  • Summary: 50%
  • Presentation: 30%
  • Annotated slides: 20%
  • Bonus points for solid discussion management


Each summary should be sent to the mailing list address gambi AT st.cs.uni-saarland.de

The Dealine for the submission is: Tuesday midnight (11:59pm) before the corresponding meeting.

You can find some suggestions on how to write a summary here.


At the end of each discussion session we will provide feedback on your summaries and suggestions on how to improve. On top of that, we will also provide a mini grade which will either be a

  • + Very well done summary, and interesting questions
  • 0 Good summary. Fulfills all the requirements
  • - Summary is lacking important aspects. Should be improved

In Class Presentations

Students pick a paper and must prepare a presentation to introduce the paper in class. The same student, with the help of the lecturer, will lead the discussion about the paper. If a topic remains un-assigned, the lecturer will pick randomly a student for preparing the presentation.

Additional Items

For a lively discussion, all the students MUST prepare a short list of questions (min 2, max 5). Questions shall be included in the summary report.

Finally, everybody shall provide a list of related papers (min 2, max 5). These papers can be used by others to known about alternative solutions to the same problem, or get a better understanding of the topic. Each of these paper must come with a brief explanation of the why you pick them; something like a 3-sentence paragraph for each paper is sufficient and must be included in the summary.


Impressum Datenschutzerklärung

<webmaster@st.cs.uni-saarland.de> · http://www.st.cs.uni-saarland.de/edu/cloudtesting/2015/ · Stand: 2018-04-05 13:40