MetaSel - Meta-learning & Algorithm Selection
Sponsors: AI Journal
- Tutorial slides (version 7.9.; also available from Tutorial page)
13h20 Bernd Bischl, LMU Munich, Germany
Applying Model-Based Optimization to Hyperparameter Optimization in Machine Learning
14h Bernhard Pfahringer, University of Waikato, New Zealand
On a few recent developments in Meta-Learning for Algorithm Ranking and Selection
All oral presentations will be relatively short (10 min.). They will be supplemented by a poster session.
14h40 Antoine Adam and Hendrik Blockeel:
Dealing with overlapping clustering: a constraint-based approach to algorithm selection
14h50 Martin Wistuba, Nicolas Schilling and Lars Schmidt-Thieme:
Learning Data Set Similarities for Hyperparameter Optimization Initializations
15h Iván Olier, Crina Grosan, Noureddin Sadawi, Larisa Soldatova and Ross King:
Meta-QSAR: learning how to learn QSARs
15h10 Tomas Kren, Martin Pilat, Klara Peskova and Roman Neruda:
Generating Workflow Graphs Using Typed Genetic Programming
15h20 Michael Smith, Tony Martinez and Christophe Giraud-Carrier:
The Potential Benefits of Data Set Filtering and Learning Algorithm Hyperparameter Optimization
15h30 Coffee break + Poster session I
The accompanying posters will be presented in a subsequent poster session.
16h Toon Van Craenendonck and Hendrik Blockeel:
Limitations of Using Constraint Set Utility in Semi-Supervised Clustering
16h10 Salisu Abdulrahman, Pavel Brazdil, Jan Rijn and Joaquin Vanschoren:
Algorithm Selection via Metalearning and Sample-based Active Testing
16h20 Catarina Felix, Carlos Soares and Alípio Jorge:
Metalearning for multiple-domain Transfer Learning
16h30 Rafael Mantovani, André Rossi, Joaquin Vanschoren and André Carvalho:
Meta-learning Approach for the Recommendation of Default Hyper-parameters Values for SVMs
16h40 Jan N. van Rijn and Joaquin Vanschoren:
RapidMiner Integration and Workflow Support for OpenML
16h50 Milan Vukicevic, Sandro Radovanovic, Joaquin Vanschoren, Giulio Napolitano and Boris Delibasic:
Towards a Collaborative Platform for Advanced Meta-Learning in Health-care Predictive Analytics
17h - 17h30 Poster session II
(The posters can be presented in any of the two poster sessions)
Patryk Kiepas, Szymon Bobek and Grzegorz J Nalepa:
Concept of rule-based configurator for Auto-WEKA using OpenML
Dirk Schäfer and Eyke Hüllermeier:
Preference-Based Meta-Learning using Dyad Ranking: Recommending Algorithms in Cold-Start Situations
Alexey Zabashta, Ivan Smetannikov and Andrey Filchenkov:
Study on meta-learning approach application in rank aggregation algorithm selection
The participants who would wish to print the poster in Porto (and avoid taking the poster on the plane), will be able to do so. There are various places where the poster can be printed from pdf, such as Grafipronto that is conveniently located at Shopping Cidade do Porto near several hotels (e.g. Tuela, Ipanema etc.). It is open all days (incl. Sunday) till 23h. See www.grafipronto.pt for details.
Do not forget: 17h45 ECML/PKDD Conference opening
Objectives of the workshop
This workshop will provide a platform for discussing the recent developments in the area of algorithm selection and configuration which arises in many diverse domains, such as machine learning, data mining, optimization and satisfiability solving.
Algorithm Selection and configuration are increasingly relevant today. Researchers and practitioners from all branches of science and technology face a large choice of parameterized machine learning algorithms, with little guidance as to which techniques to use. Moreover, data mining challenges frequently remind us that algorithm selection and configuration are crucial in order to achieve the best performance, and drive industrial applications.
Meta-learning leverages knowledge of past algorithm applications to select the best techniques for future applications, and offers effective techniques that are superior to humans both in terms of the end result and especially in the time required to achieve it. In this workshop we will discuss different ways of exploiting meta-learning techniques to identify the potentially best algorithm(s) for a new task, based on meta-level information and prior experiments. We also discuss the prerequisites for effective meta-learning systems, for example infrastructure such as OpenML.org.
Many contemporary problems also require that solutions be elaborated in the form of complex systems or workflows which include many different processes or operations. Constructing such complex systems or workflows requires extensive expertise, and could be greatly facilitated by leveraging planning, meta-learning and intelligent system design. This task is inherently interdisciplinary, as it builds on expertise in various areas of AI.
The workshop will take place 7 September (Monday), just before the main conference. It will be preceded by a tutorial on the same topic in the morning (9h-12h) for the benefit of participants of the workshop who have less experience in the area. See our tutorial page for more details.
This workshop is a follow-up on the successful workshop MetaSel-2014 that was associated with ECAI 2014 in Prague (metasel2014.inescporto.pt). Last year’s workshop had 3 invited talks and 12 other presentations. It had more than 20 participants and all contributions can be found in the CEUR proceedings (http://ceur-ws.org/Vol-1201/). The proposed workshop is also closely related to PlanLearn-2012 (http://datamining.liacs.nl/planlearn.html), which took place at ECAI 2012, and the AutoML workshop, which will take place at ICML 2015.
Authors working on the following topics are invited to submit a paper to the workshop. This list is not exhaustive, and other topics, strongly associated with algorithm selection and meta-learning, will be considered.
- Algorithm / Model selection and configuration
- Meta-learning and exploitation of meta-knowledge
- Experimentation and evaluation of learning processes
- Hyper-parameter optimization
- Planning to learn and to construct workflows
- Applications of workflow planning
- Exploitation of ontologies of tasks and methods
- Exploitation of benchmarks and experimentation
- Representation of learning goals and states in learning
- Control and coordination of learning processes
- Layered learning
- Multi-task and transfer learning
- Learning to learn
- Intelligent design
- Performance modeling and process mining
Submissions and Review Process
- (Paper submission deadline: Monday, June 29, 2015)
- (Paper acceptance notification: Tuesday, July 21, 2015)
- (Paper camera-ready deadline: Friday August 7, 2015)
Submissions are possible either as a full paper or as an extended abstract. Full papers can consist of a maximum of 12 pages, extended abstracts up to 2 pages, in the Springer format. Full papers should present more advanced work, covering research or applications. Extended abstracts may present current, recently published or future research, and can cover a wider scope. For instance, they may be position statements, offer a specific scientific or business problem to be solved by machine learning (ML) / data mining (DM) or describe a demo or installation.
Each submission must be submitted online via the Easychair submission interface (https://easychair.org/conferences/?conf=metasel2015). Submissions can be updated at will before the submission deadline. The only accepted format for submitted papers is PDF.
Each paper submission will be evaluated on the basis of relevance, significance of contribution, technical quality, scholarship, and quality of presentation, by at least two members of the program committee. The evaluation process will be single blind (the names of the reviewers will not be disclosed). All accepted submissions will be included in the workshop proceedings (as CEUR Workshop Proceedings). Electronic versions of accepted submissions will also be made publicly available on the conference web site. At least one author of each accepted full paper or extended abstract is required to attend the workshop to present the contribution.
A selection of accepted papers will be presented in the plenary session. The remainder of accepted submissions will be presented in the form of short talks and a poster session. The papers selected for plenary presentation will be identified in the proceedings.
- Pavel Brazdil, FEP, Univ. of Porto / Inesc Tec, Portugal, pbrazdil at inescporto.pt
- Joaquin Vanschoren, Eindhoven Univ. of Technology (TU/e), Eindhoven, The Netherlands, j.vanschoren at tue.nl
- Lars Kotthoff, University of British Columbia, Vancouver, Canada, larsko at cs.ubc.ca
- Christophe Giraud-Carrier, Brigham Young Univ., USA
- Pavel Brazdil, LIAAD-INESC TEC / FEP, University of Porto, Portugal
- André C. P. Carvalho, USP, Brasil
- Claudia Diamantini, Università Politecnica delle Marche, Italy
- Johannes Fuernkranz, TU Darmstadt, Germany
- Christophe Giraud-Carrier, Brigham Young Univ., USA
- Krzysztof Grabczewski, Nicolaus Copernicus University, Poland
- Frank Hutter, University of Freiburg, Germany
- Christopher Jefferson, University of St Andrews, UK
- Alexandros Kalousis, U Geneva, Switzerland
- Jörg-Uwe Kietz, U.Zurich, Switzerland
- Lars Kotthoff, University of British Columbia, Canada
- Yuri Malitsky, IBM Research, USA
- Bernhard Pfahringer, U Waikato, New Zealand
- Vid Podpecan, Jozef Stefan Institute, Slovenia
- Ricardo Prudêncio, Univ. Federal de Pernambuco Recife (PE), Brasil
- Samantha Sanders, Brigham Young University, USA
- Michael Smith, Brigham Young University, USA
- Carlos Soares, FEUP, University of Porto, Portugal
- Guido Tack, Monash University, Australia
- Joaquin Vanschoren, U. Leiden / KU Leuven
- Ricardo Vilalta, University of Houston, USA
- Filip Železný, CVUT, Prague, R.Checa