Skip to main content
Department of Information Technology

SysCon machine learning reading group

A weekly machine learning reading group. Each session is 1h. Sessions are typically on Thursdays. Each week we take turn to pick a paper to read, which we then go through together and discuss.

Contact person: Daniel Gedon.

Detailed internal information.

Paper List 2023

Week Paper Venue
3 Out-Of-Distribution Detection Is Not All You Need AAAI 2023
4 Gradient Descent Happens in a Tiny Subspace arxiv, 2018
5 Everything is Connected: Graph Neural Networks arxiv, 2023
6 The Forward-Forward Algorithm: Some Preliminary Investigations arxiv, 2022
7 Neural Networks Trained with SGD Learn Distributions of Increasing Complexity arxiv, 2022
8 Spectral Temporal Graph Neural Network for Multivariate Time-series Forecasting NeurIPS 2020
9 Unsupervised Out-of-Distribution Detection with Diffusion Inpainting arxiv, 2023
10 How Powerful are Graph Neural Networks? ICLR 2019
11 Why AI is Harder Than We Think arxiv, 2021
12 Resurrecting Recurrent Neural Networks for Long Sequences arxiv, 2023
13 PID-GAN: A GAN Framework based on a Physics-informed Discriminator for Uncertainty Quantification with Physics KDD 2021
14 Evaluating the Fairness of Deep Learning Uncertainty Estimates in Medical Image Analysis MIDL 2023
15 (Easter) -
16 A Deep Conjugate Direction Method for Iteratively Solving Linear Systems arxiv, 2022
17 Assaying Out-Of-Distribution Generalization in Transfer Learning NeurIPS 2022
18 I2SB: Image-to-Image Schrödinger Bridge ICML 2023
19 High-Resolution Image Synthesis with Latent Diffusion Models CVPR 2022
20 - -
21 Consistency Models ICML 2023
22 Simplified State Space Layers for Sequence Modeling ICLR 2023
23 Mechanism of Feature Learning in Deep Fully Connected Networks and Kernel Machines that Recursively Learn Features arxiv, 2022
24 Transport with Support: Data-Conditional Diffusion Bridges arxiv, 2023
-
(Summer break)
-
34 Loss Landscapes are All You Need: Neural Network Generalization Can Be Explained Without the Implicit Bias of Gradient Descent ICLR 2023
35 A Law of Data Separation in Deep Learning PNAS, 2023
36 - -
37 Efficient Formal Safety Analysis of Neural Networks NeurIPS 2018
38 Self-Supervised Learning with Data Augmentations Provably Isolates Content from Style NeurIPS 2021
39 - -
40 - -
41 - -
42 - -
43 - -
44 A Framework for Understanding Sources of Harm throughout the Machine Learning Life Cycle EAAMO 2021
45 Considerations for Addressing Bias in Artificial Intelligence for Health Equity npj Digital Medicine, 2023
46 Large Language Models Propagate Race-based Medicine npj Digital Medicine, 2023
47 Algorithmic Fairness In Artificial Intelligence For Medicine And Healthcare Nature Biomedical Engineering, 2023
48 Reclaiming AI as a theoretical tool for cognitive science PsyArXiv
49 On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? *parrot emoji* FAccT, 2021

Paper List 2022

Week Paper Venue
7 Energy-based Out-of-distribution Detection NeurIPS 2020
8 Deep Learning Through the Lens of Example Difficulty NeurIPS 2021
9 Transformers Can Do Bayesian Inference ICLR 2022
10 The Deep Bootstrap Framework: Good Online Learners are Good Offline Generalizers ICLR 2021
11 Comparing Elementary Cellular Automata Classifications with a Convolutional Neural Network ICAART 2021
12 Random Synaptic Feedback Weights Support Error Backpropagation for Deep Learning Nature Communications, 2016
13 Selective Classification for Deep Neural Networks NeurIPS 2017
14 Uncalibrated Models Can Improve Human-AI Collaboration arxiv, 2022
15 Conformalized Quantile Regression NeurIPS 2019
16 - -
17 - -
18 As simple as ABC: a straightforward method for evaluating performance of machine learning models for classification -
19 - -
20 Open-Set Recognition: a Good Closed-Set Classifier is All You Need? ICLR 2022
21 When are Bayesian model probabilities overconfident? arxiv, 2020
22 Shaking the foundations: delusions in sequence models for interaction and control arxiv, 2021
23 Weakly-Supervised Disentanglement Without Compromises ICML 2020
24 Greedy Bayesian Posterior Approximation with Deep Ensembles TMLR, 2022
25 Linear Time Sinkhorn Divergences using Positive Features NeurIPS 2020
-
(Summer break)
-
35 On the Information Bottleneck Theory of Deep Learning ICLR 2018
36 Reliable and Trustworthy Machine Learning for Health Using Dataset Shift Detection NeurIPS 2021
37 Learning to learn by gradient descent by gradient descent NeurIPS 2016
38 Adversarial Examples Are Not Bugs, They Are Features NeurIPS 2019
39 Generative Modeling by Estimating Gradients of the Data Distribution NeurIPS 2019
40 Mechanistic models versus machine learning, a fight worth fighting for the biological community? Biology Letters, 2018
41 RankFeat: Rank-1 Feature Removal for Out-of-distribution Detection NeurIPS 2022
42 Pseudo-Spherical Contrastive Divergence NeurIPS 2021
43 Multi-scale Feature Learning Dynamics: Insights for Double Descent ICML 2022
44 Fine-Tuning can Distort Pretrained Features and Underperform Out-of-Distribution ICLR 2022
45 Learning deep representations by mutual information estimation and maximization ICLR 2019
46 Prioritized Training on Points that are learnable, Worth Learning, and Not Yet Learnt ICML 2022
47 Collocation based training of neural ordinary differential equations Statistical Applications in Genetics and Molecular Biology, 2021
48 Denoising Diffusion Implicit Models ICLR 2021
49 Physics-Informed Neural Networks for Cardiac Activation Mapping Frontiers in Physics, 2020
50 Continuous Time Analysis of Momentum Methods JMLR, 2020

Paper List 2021

Week Paper Venue
2 Transformers are RNNs: Fast Autoregressive Transformers with Linear Attention ICML 2020
3 Getting a CLUE: A Method for Explaining Uncertainty Estimates ICLR 2021
4 No MCMC for Me: Amortized Sampling for Fast and Stable Training of Energy-Based Models ICLR 2021
5 Meta Pseudo Labels CVPR 2021
6 On the Origin of Implicit Regularization in Stochastic Gradient Descent ICLR 2021
7 Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision ICML 2021
8 Neural Relational Inference for Interacting Systems ICML 2018
9 Infinitely Deep Bayesian Neural Networks with Stochastic Differential Equations AISTATS 2022
10 Unsupervised Learning of Visual Features by Contrasting Cluster Assignments NeurIPS 2020
11 Gradient Descent on Neural Networks Typically Occurs at the Edge of Stability ICLR 2021
12 Your GAN is Secretly an Energy-based Model and You Should use Discriminator Driven Latent Sampling NeurIPS 2020
13 Loss Surface Simplexes for Mode Connecting Volumes and Fast Ensembling ICML 2021
14 Q-Learning in enormous action spaces via amortized approximate maximization arxiv, 2020
15 Learning Mesh-Based Simulation with Graph Networks ICLR 2021
16 - -
17 Stiff Neural Ordinary Differential Equations arxiv, 2021
18 PixelTransformer: Sample Conditioned Signal Generation ICML 2021
19 - -
20 - -
21 - -
22 - -
23 Meta-Learning Bidirectional Update Rules ICML 2021
24 Deconstructing the Regularization of BatchNorm ICLR 2021
-
(Summer break)
-
34 DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation CVPR 2019
35 Differentiable Particle Filtering via Entropy-Regularized Optimal Transport ICML 2021
36 Revisiting the Calibration of Modern Neural Networks NeurIPS 2021
37 NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis ECCV 2020
38 Hierarchical VAEs Know What They Don't Know ICML 2021
39 Information Dropout: Learning Optimal Representations Through Noisy Computation arxiv, 2016
40 SMD-Nets: Stereo Mixture Density Networks CVPR 2021
41 Learning to Simulate Complex Physics with Graph Networks ICML 2020
42 Grokking: Generalization Beyond Overfitting on Small Algorithmic Datasets ICLR Workshops 2021
43 Deep Classifiers with Label Noise Modeling and Distance Awareness TMLR, 2022
44 - -
45 Transferring Inductive Biases through Knowledge Distillation arxiv, 2020
46 Masked Autoencoders Are Scalable Vision Learners arxiv, 2021
47 On the Importance of Gradients for Detecting Distributional Shifts in the Wild NeurIPS 2021
48 An Information-theoretic Approach to Distribution Shifts NeurIPS 2021
49 Periodic Activation Functions Induce Stationarity NeurIPS 2021
50 Efficiently Modeling Long Sequences with Structured State Spaces ICLR 2021

Paper List 2020

Week Paper Venue
2 Z-Forcing: Training Stochastic Recurrent Networks NeurIPS 2017
3 Multiplicative Interactions and Where to Find Them ICLR 2020
4 A Primal-Dual link between GANs and Autoencoders NeurIPS 2019
5 Modelling heterogeneous distributions with an Uncountable Mixture of Asymmetric Laplacians NeurIPS 2019
6 Uncertainty Decomposition in Bayesian Neural Networks with Latent Variables arxiv, 2017
7 Decomposition of Uncertainty in Bayesian Deep Learning for Efficient and Risk-sensitive Learning ICML 2018
8 Convolutional Conditional Neural Processes ICLR 2020
9 Bayesian Deep Learning and a Probabilistic Perspective of Generalization arxiv, 2020
10 Batch Normalization Biases Deep Residual Networks Towards Shallow Paths NeurIPS 2020
11 - -
12 Conservative Uncertainty Estimation By Fitting Prior Networks ICLR 2020
13 Pitfalls of In-Domain Uncertainty Estimation and Ensembling in Deep Learning ICLR 2020
14 Normalizing Flows: An Introduction and Review of Current Methods TPAMI, 2020
15 Beyond temperature scaling: Obtaining well-calibrated multiclass probabilities with Dirichlet calibration NeurIPS 2019
16 How Good is the Bayes Posterior in Deep Neural Networks Really? ICML 2020
17 - -
18 - -
19 Stable Neural Flows arxiv, 2020
20 Your Classifier is Secretly an Energy Based Model and You Should Treat it Like One ICLR 2020
21 - -
22 BatchEnsemble: An Alternative Approach to Efficient Ensemble and Lifelong Learning ICLR 2020
23 Efficient and Scalable Bayesian Neural Nets with Rank-1 Factors ICML 2020
24 End-to-End Object Detection with Transformers ECCV 2020
25 Joint Training of Variational Auto-Encoder and Latent Energy-Based Model CVPR 2020
-
(Summer break)
-
36 Denoising Diffusion Probabilistic Models NeurIPS 2020
37 Gated Linear Networks arxiv, 2020
38 Uncertainty Estimation Using a Single Deep Deterministic Neural Network ICML 2020
39 Simple and Principled Uncertainty Estimation with Deterministic Deep Learning via Distance Awareness NeurIPS 2020
40 - -
41 Satellite Conjunction Analysis and the False Confidence Theorem 2019
42 Implicit Gradient Regularization ICLR 2021
43 - -
44 - -
45 Approximate Inference Turns Deep Networks into Gaussian Processes NeurIPS 2019
46 VAEBM: A Symbiosis between Variational Autoencoders and Energy-based Models ICLR 2021
47 Very Deep VAEs Generalize Autoregressive Models and Can Outperform Them on Images ICLR 2021
48 Rethinking Attention with Performers ICLR 2021
49 - -
50 Dissecting Neural ODEs NeurIPS 2020
51 Score-Based Generative Modeling through Stochastic Differential Equations ICLR 2021

Paper List 2019

Week Paper Venue
2 Neural Processes ICML Workshops 2018
3 How Does Batch Normalization Help Optimization? NeurIPS 2018
4 An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling arxiv, 2018
5 A Complete Recipe for Stochastic Gradient MCMC NeurIPS 2015
6 Deep Reinforcement Learning in a Handful of Trials using Probabilistic Dynamics Models NeurIPS 2018
7 A Simple Baseline for Bayesian Uncertainty in Deep Learning NeurIPS 2019
8 Fine-Grained Analysis of Optimization and Generalization for Overparameterized Two-Layer Neural Networks ICML 2019
9 Language Models are Unsupervised Multitask Learners 2019
10 Coupled Variational Bayes via Optimization Embedding NeurIPS 2018
11 A recurrent neural network without chaos ICLR 2017
12 Evaluating model calibration in classification AISTATS 2019
13 Generating High Fidelity Images with Subscale Pixel Networks and Multidimensional Upscaling ICLR 2019
14 Stochastic Gradient Descent as Approximate Bayesian Inference JMLR, 2017
15 - -
16 - -
17 - -
18 Visualizing the Loss Landscape of Neural Nets NeurIPS 2018
19 - -
20 - -
21 - -
22 Attention Is All You Need NeurIPS 2017
23 LaserNet: An Efficient Probabilistic 3D Object Detector for Autonomous Driving CVPR 2019
-
(Summer break)
-
40 Trellis Networks for Sequence Modeling ICLR 2019
41 Variational Inference with Normalizing Flows ICML 2015
42 Improving Variational Inference with Inverse Autoregressive Flow NeurIPS 2016
43 Neural Autoregressive Flows ICML 2018
44 - -
45 Weight Uncertainty in Neural Networks ICML 2015
46 Learning nonlinear state-space models using deep autoencoders CDC 2018
47 Learning Latent Dynamics for Planning from Pixels ICML 2019
48 Dream to Control: Learning Behaviors by Latent Imagination ICLR 2020
Updated  2024-03-02 09:21:20 by Daniel Gedon.