Welcome

“Learning to Quantify: Methods and Applications” is a tutorial + workshop event co-located with the ECML/PKDD 2024 conference, and will take place on Friday, September 13, 2024, in Vilnius, Lithuania.

Learning to Quantify (LQ - also known as “quantification“, or “supervised prevalence estimation“, or “class prior estimation“, or “unfolding”) is the task of training class prevalence estimators via supervised learning. In other words, the task of these trained models is to estimate, given an unlabelled sample of data items and a set of classes, the prevalence (i.e., relative frequency) of each such class in the sample.

LQ is interesting in all applications of classification in which the final goal is not determining which class (or classes) individual unlabelled data items belong to, but estimating the percentages of data items that belong to the classes of interest, i.e., estimating the distribution of the unlabelled data items across the classes. Example disciplines whose interest in labelling data items is at the aggregate level (rather than at the individual level) are the social sciences, political science, market research, ecological modelling, and epidemiology.

While LQ may in principle be solved by classifying each data item in the sample and counting how many such items have been labelled with a certain class, it has been shown that this “classify and count” method yields suboptimal quantification accuracy. As a result, quantification is now no longer considered a mere byproduct of classification, and has evolved as a task of its own.

The goal of the Learning to Quantify tutorial is to provide an introduction to Learning to Quantify that presents its rationale, applications, methods, and evaluation measures and methodologies, to researchers who might want to become up-to-speed on this topic. A book on Learning to Quantify will also be distributed to participants, along with the slides used by the instructors.

The goal of the LQ 2024 workshop is to bring together all researchers interested in methods, algorithms, evaluation measures, evaluation protocols, and methodologies for LQ, as well as practitioners interested in the practical application of the above to managing large quantities of data. The workshop will feature presentations of submitted papers, presentations of the results of the 2024 Learning to Quantify data challenge, and a final open discussion on “what’s next in Learning to Quantify”.

LQ 2024 is supported by the SoBigData++ project, funded by the European Commission (Grant 871042) under the H2020 Programme INFRAIA-2019-1, by the AI4Media project, funded by the European Commission (Grant 951911) under the H2020 Programme ICT-48-2020, and by the QuaDaSh project, funded by the European Commission under the NextGenerationEU program. The organizers’ opinions do not necessarily reflect those of the European Commission.

ai4media logo sobigdata logo

Call for papers for the LQ 2024 Workshop

We seek papers on any of the following topics, which will form the main themes of the LQ 2024 workshop:

  • Binary, multiclass, multilabel, and ordinal LQ
  • Supervised algorithms for LQ
  • Semi-supervised / transductive LQ
  • Deep learning for LQ
  • Representation learning for LQ
  • LQ and dataset shift
  • Evaluation measures for LQ
  • Experimental protocols for the evaluation of LQ
  • Quantification of streaming data
  • Cost-sensitive quantification
  • Improving classifier performance via LQ
  • New datasets for evaluating quantification systems
  • Novel applications of LQ

and other topics of relevance to LQ. Two categories of papers are of interest:

  • papers reporting original, unpublished research;
  • papers {published in 2024 / currently under submission / accepted in 2024} at other {workshops / conferences / journals}, provided this double submission does not violate the rules of these {workshops / conferences / journals}.
Submission

Papers should be submitted (specifying which of the two above categories they belong to) via EasyChair.

Papers should be formatted according to Springer’s LNCS template, and should be up to 16 pages (including references) in length; however, this is just the upper bound, and contributions of any length up to this bound will be considered.

Other information

At least one author of each accepted paper must register to present the work. The workshop will be a hybrid event, but it is strongly recommended that authors of accepted papers present the work in-presence. The proceedings of the workshop will not be formally published, so as to allow authors to resubmit their work to other conferences. Informal proceedings will be published on the workshop website; however, for each accepted paper, it will be left at the discretion of the authors to decide whether to contribute their paper or not to these proceedings.

Important dates (all deadlines are 23:59 AoE)
  • Paper submission deadline: June 15, 2024
  • A/R notification deadline: July 15, 2024
  • Final copy submission deadline: August 25, 2024
  • Workshop: September 13, 2024

Chairs

Mirko Bunse

Mirko Bunse (tutorial speaker and workshop organizer)

Artificial Intelligence Group, TU Dortmund University, Germany

Pablo González

Pablo González (workshop organizer)

Artificial Intelligence Center, University of Oviedo, Spain

Alejandro Moreo

Alejandro Moreo (tutorial speaker and workshop organizer)

Istituto di Scienza e Tecnologie dell’Informazione, Consiglio Nazionale delle Ricerche, Pisa, Italy

Fabrizio Sebastiani

Fabrizio Sebastiani (tutorial speaker and workshop organizer)

Istituto di Scienza e Tecnologie dell’Informazione, Consiglio Nazionale delle Ricerche, Pisa, Italy

Program Committee
  • Rocío Alaíz-Rodríguez, University of León, ES
  • Gustavo Batista, University of New South Wales, AU
  • Juan José del Coz, University of Oviedo, ES
  • Andrea Esuli, Consiglio Nazionale delle Ricerche, IT
  • Alessandro Fabris, Università di Padova, IT
  • Cèsar Ferri, Universitat Politècnica de València, ES
  • George Forman, Amazon Research, US
  • Wei Gao, Singapore Management University, SG
  • Rafael Izbicki, Federal University of São Carlos, BR
  • André G. Maletzke, Universidade Estadual do Oeste do Paraná, BR
  • Tobias Schumacher, University of Mannheim, DE
  • Marco Saerens, Catholic University of Louvain, BE
  • Dirk Tasche, Swiss Financial Market Supervisory Authority, CH

Program

The following is a preliminary program of the LQ 2024 Tutorial+Workshop event.

When: Friday, September 13, 2024

Where: Beta 1

09:00 11:00 Tutorial: Learning to Quantify, Part I, by Mirko Bunse, Alejandro Moreo and Fabrizio Sebastiani
11:00 11:20 Coffee Break
11:20 13:00 Tutorial: Learning to Quantify, Part II, by Mirko Bunse, Alejandro Moreo and Fabrizio Sebastiani
13:00 14:00 Lunch Break
Chair: Mirko Bunse 14:20 14:30 Workshop Chairs' Introduction
Chair: Mirko Bunse 14:30 14:50 Comments on Friedman's Method for Class Distribution Estimation, by Dirk Tasche
14:50 15:10 Quantification Over Time, by Feiyu Li, Hassan H. Gharakheili and Gustavo Batista
15:10 15:30 Enhancing Quantification through Meta-Learning, by Guilherme Gomes, Willian Zalewski and Andre Maletzke
Chair: Mirko Bunse 15:30 15:50 An Overview of LeQua 2024, by Andrea Esuli, Alejandro Moreo, Fabrizio Sebastiani and Gianluca Sperduti
16:00 16:20 Coffee Break
Chair: Alejandro Moreo 16:20 16:30 Evaluating Continuous Sweep and Comparison Using Underlying Classifiers, by Kevin Kloos. UniLeiden at LeQua2024
16:30 16:40 Ensemble Learning to Quantify, by Zahra Donyavi, Feiyu Li and Gustavo Batista. The CSE UNSW Team at LeQua2024
16:40 16:50 Regularized Soft-Max Likelihood Maximization, by Mirko Bunse and Tobias Lotz. Lamarr at LeQua2024
16:50 17:00 Quantification via Gaussian Latent Space Representations., by Pablo González and Olaya Pérez. UniOvi at LeQua2024
17:00 17:30 LeQua joint discussion
Chair: Alejandro Moreo 17:30 18:00 Open discussion