Summer Vision Reading Group

Who: We're a bunch of PhD students across the CS, CE, and EE programs at ASU, UF, and Rice.
What: We read concepts in machine learning, and brainstorm their applications in frontier problems in computer vision, but not limited to vision.
When, Where, How: Send me or Joshua Feinglass an email if you want to join us! Zoom Every Saturday 1300hrs MST

2021 Archive

Date Host Topic Paper(s) Read Participants Notes
May 8, 2021 Tejas Gokhale Uncertainty Sets for Image Classification
  • Uncertainty Sets for Image Classifiers using Conformal Prediction, Angelopouls et al. ICLR 2021, pdf
Sheng Cheng, Joshua Feinglass, Tejas Gokhale, Blake Harrison, Ishan Khurjekar, Yiran Luo uncertainty, prediction sets vs single prediction, coverage
May 15, 2021 Blake Harrison Imagination in Navigation
Sheng Cheng, Joshua Feinglass, Tejas Gokhale, Blake Harrison, Yiran Luo wave function collapse, exploration, imagination, voxels and graphs
May 22, 2021 Yiran Luo CLIP
  • Learning Transferable Visual Models From Natural Language Supervision, Radford et al. ICML 2021, pdf
May 29, 2021 Tejas Gokhale Implicit Neural Representations
  • Implicit Neural Representations with Periodic Activation Functions, Sitzmann et al. NeurIPS 2020 pdf
  • Adversarial Generation of Continuous Images, Skorokhodov et al. CVPR 2021, pdf
first in-person meeting of the group. proof
June 12, 2021 Sheng Cheng Part-Whole Hierarchies
  • How to Represent Part-Whole Hierarchies in a Neural Network, Geoff Hinton, preprint. pdf
June 23, 2021 Joshua Feinglass Evaluation Metrics I (Image Captioning)
  • METEOR: An Automatic Metric for MT Evaluation with Improved Correlation with Human Judgments, Banerjee et al. ACL Workshops 2005. pdf
  • SPICE: Semantic Propositional Image Caption Evaluation, Anderson et al. ECCV 2016, pdf
  • CIDEr: Consensus-based Image Description Evaluation, Vedantam et al. CVPR 2015, pdf.
June 30, 2021 Blake Harrison Evolutionary Strategies
July 14, 2021 Tejas Gokhale Test-Time Training
  • Test-Time Training with Self-Supervision for Generalization under Distribution Shifts, Sun et al. ICML 2020, pdf.
  • Tent: Fully Test-time Adaptation by Entropy Minimization, Wang et al. ICLR 2021, pdf
July 21, 2021 Albert Reed Neural Radiance Fields
  • acorn: Adaptive Coordinate Networks for Neural Scene Representation, Martel et al. ACM ToG 2021, pdf
  • BARF : Bundle-Adjusting Neural Radiance Fields, Lin et al. ICCV 2021, pdf
July 28, 2021 David Ramirez Evaluation Metrics II (Text Generation)
  • All That’s ‘Human’ Is Not Gold: Evaluating Human Evaluation of Generated Text, Clark et al. ACL 2021, pdf
Aug 4, 2021 Blake Harrison RL for Game Playing
  • Mastering Atari, Go, Chess and Shogi by Planning with a Learned Model, Schrittwieser et al. Nature 2020, pdf
Aug 11, 2021 Embodied Perception
  • Gibson Env: Real-World Perception for Embodied Agents, Xia et al. CVPR 2018, pdf
Aug 18, 2021 Video Feature Extraction
  • Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset, Carreira et al. CVPR 2017 pdf
  • SlowFast Networks for Video Recognition, Feichtenhofer al. Tech Report, pdf

2020 Archive

Date Host Topic Paper(s) Read Participants Notes
May 16, 2020 Tejas Gokhale Curriculum Learning
  • Curriculum Learning, Bengio et al. ICML 2009, pdf
  • On The Power of Curriculum Learning in Training Deep Networks, Hacohen & Weinshall, ICML 2019, pdf
Tejas Gokhale, Joshua Feinglass, Kowshik Thopalli, Man Luo, Zhiyuan Fang Connection with active learning, curriculum learning in GANs?, curriculum learning for domain adaptation
May 23, 2020 Kowshik Thopalli / Tejas Gokhale Active Learning
  • Two Faces of Active Learning, Dasgupta, ALT 2009 pdf
  • Active Learning for Deep Object Detection, Brust et al. 2018 pdf
Kowshik Thopalli, Bhargav Ghanekar, Ishan Khurjekar, Joshua Feinglass, Man Luo, Sheng Cheng, Tejas Gokhale Strategies for selecting samples to label, how to select the best samples that improve performance vs heuristic-based selection?, Schrodinger's Douchebags
May 30, 2020 Kowshik Thopalli Multi-modal Fusion
  • Multimodal Machine Learning:A Survey and Taxonomy, Baltrušaitis et al., TPAMI 2019 pdf
Kowshik Thopalli, Bhargav Ghanekar, Ishan Khurjekar, Joshua Feinglass, Man Luo, Sheng Cheng, Tejas Gokhale Use-cases, when is it critical, audio-video alignment, modalities at different sampling rates, self-driving,
June 6, 2020 Ishan Khurjekar Uncertainty Estimation I
  • Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles, Lakshminarayanan et al. NeurIPS 2017 pdf
Ishan Khurjekar, Bhargav Ghanekar, Joshua Feinglass, Kowshik Thopalli, Man Luo, Sheng Cheng, Tejas Gokhale Uncertainty, epistemic vs sensing, uncerrtainty estimation --> stronger metrics for evaluation of models,
June 13, 2020 Joshua Feinglass Transformers, BERT, VilBERT/LXMERT
  • Attention is All You Need, Vaswani et al. NeurIPS 2017 pdf, blog
  • BERT: Pre-training of Deep Bidirectional Transformers forLanguage Understanding, Devlin et al. NAACL 2019 pdf
  • LXMERT, Tan et al. EMNLP 2019 pdf
  • VilBERT, Lu et al, NeurIPS 2019 pdf
Joshua Feinglass, Bhargav Ghanekar, Ishan Khurjekar, Kowshik Thopalli, Man Luo, Sheng Cheng, Tejas Gokhale Attention, Self-Attention, Encoder-Decoder Architecture, Transformer Decoder, BERT pre-training tasks and intuition behind their choice, extension to cross-modal (vision+language) pre-training
June 20, 2020 Bhargav Ghanekar Computational Imaging
  • PhaseCam3D, Wu et al. ICCP 2019 pdf
  • All-optical ML using diffractive deep NN, Lin et al. Science Mag 2018 pdf
Bhargav Ghanekar, Joshua Feinglass, Ishan Khurjekar, Kowshik Thopalli, Man Luo, Sheng Cheng, Tejas Gokhale Point Spread Function, Relation between Defocus Blur and Depth, Designing Phase Masks through end-to-end learning
June 27, 2020 Ishan Khurjekar Uncertainty Estimation II
  • What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?, Kendall & Gall, NeurIPS 2017 pdf
Ishan Khurjekar, Bhargav Ghanekar, Changhoon Kim, Man Luo, Sheng Cheng, Tejas Gokhale Combining Aleatoric and Epistemic Uncertainty. Heteroscedastic Uncertainty as Learned Loss Attenuation: (1) inputs with high predicted uncertainty will have a smaller effect on the loss, (2) model is discouraged from predicting very low uncertainty for points with high residual error,
July 4, 2020 Changhoon Kim Adversarial Attack
  • Disrupting Image-Translation-Based DeepFake Algorithms with Adversarial Attacks, Yeh et al. WACV 2020 pdf
Changhoon Kim, Ishan Khurjekar, Joshua Feinglass, Kowshik Thopalli, Man Luo, Sheng Cheng, Tejas Gokhale Adversarial Attacks against malicious generative neural networks (DeepFake / DeepNude), extensions, generalization to unseen networks, ensembles
July 11, 2020 Sheng Cheng Contrastive Learning
  • Momentum Contrast for Unsupervised Visual Representation Learning, He et al. CVPR 2020 pdf
  • A Simple Framework for Contrastive Learning of Visual Representations, Chen et al. ICML 2020 pdf
Sheng Cheng, Bhargav Ghanekar, Ishan Khurjekar, Joshua Feinglass, Man Luo, Tejas Gokhale Contrastive Learning as a self-supervised learning framework, connection with hashing, auto-encoders, language pretraining, word2vec. Open discussion on many faces of generalization.
July 18, 2020 Man Luo Neuro-Symbolic Learning
  • Logic Tensor Networks: Deep Learning and Logical Reasoning from Data and Knowledge, Serafini & Garcez. 2016 pdf
  • Logic Tensor Networks for Semantic Image Interpretation, Donadello et al. IJCAI 2017 pdf
Man Luo, Sheng Cheng, Bhargav Ghanekar, Ishan Khurjekar, Joshua Feinglass, Tejas Gokhale symbolic logic, super-quick intro to knowledge representation (propositional, FOL), demo of a reasoning problem in CLINGO, logic tensor networks, open-ended discussion about application in vision-and-language.
July 25, 2020 Tejas Gokhale Reinforcement Learning
  • Tutorial: Deep Reinforcement Learning, David Silver. ICML 2016 pdf
  • Playing Atari with Deep Reinforcement Learning, Mnih et al. NIPS 2013, DL Workshop
Tejas Gokhale, Sheng Cheng, Bhargav Ghanekar, Ishan Khurjekar, Joshua Feinglass, Man Luo RL vocabulary: state, action, reward, policy, discount factor. intuition behind experential replay and discounted reward, simpler example: navigation from (0, 0) to (10, 10). SARSA. Breakthrough in learning to play Atari games.
August 1, 2020 Joshua Feinglass Information Theory
  • A Gentle Tutorial on Information Theory and Learning, Roni Rosenfeld, CMU 1999, (CMU 11761 pdf
  • Information Theory, Cosma Shalizi, Complex Systems Summer School 2010 pdf
  • InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets, Chen et al. NeurIPS 2016. pdf
Joshua Feinglass, Sheng Cheng, Ishan Khurjekar, Man Luo, Tejas Gokhale Definitions of information, mututal information, entropy, Fischer Information, random thoughts about Huffmann coding and Wavelet compression, InfoGAN, conditional GAN, pitfalls and tricks-of-the-trade for GAN training, mode collapse
August 8, 2020 - cancelled
August 15, 2020 Man Luo Information Retrieval Man Luo, Sheng Cheng, Joshua Feinglass, Tejas Gokhale, Ishan Khurjekar, Yiran Luo
August 22, 2020 Tejas Gokhale Convex Optimization
  • Intro to Convex Optimization for ML, John Duchi, Stanford CS294, pdf
  • Input Convex Neural Networks, Amos et al. ICML 2017, pdf
Tejas Gokhale, Sheng Cheng, Joshua Feinglass, Yiran Luo, Kuntal Pal. Definitions of convex sets, functions. The general optimization problem. The convex optimization problem. Why is convexity nice? Lgrangian duality, steepest descent, gradient descent, proximal methods (projected GD), Newton.
Input convex NN (convex inference)
August 29, 2020 Ishan Khurjekar Graph Neural Networks
  • Representation Learning on Networks Tutorial, Jure Leskovec, WWW18 pdf
  • The graph neural network model, Scarselli et al. IEEE-TNN2009 pdf
  • Semi-Supervised Classification with Graph Convolutional Networks, Kipf & Welling, ICLR 2017 pdf
Ishan Khurjekar, Man Luo, Yiran Luo, Sheng Cheng, Joshua Feinglass, Tejas Gokhale, Amrita Bhattacharjee, Weidong Zhang Graph neural networks, aggregation functions, training (supervises/unsupervised), some applications, aggregation as convolution.
September 5, 2020 - cancelled
September 12, 2020 Tejas Gokhale Causal Inference
  • Causal Inference: A Tutorial, Fan Li, Duke University pdf
  • Amortized learning of neural causal representations, Ke et al. ICLR2020 CL Workshop pdf
TG,BG,IK,JF,YL,SC,ML Causality notations: variables, interventions/treatments, outcomes, confounders, CI as a missing data problem,
Learning causal models with NN, results on synthetic data, missing pieces/restrictive assumptions for real-world data
September 19, 2020 Joshua Feinglass (transfer hosting)/ Kowshik Thopalli (supervision) / Tejas Gokhale (meta-hosting) Meta Learning
  • Meta Learning Tutorial, Finn&Levine, ICML 2019 link
  • MAML, Finn et al. ICML2017 pdf
JF,KT,TG,BG,IK,YL,SC,ML,AB Learning, learning to learn, meta-train, meta-test, MAML algo discussion, use-cases, extensions with "unequal", "hierarchical", "unrelated" tasks, insights from KT about faster algos, taskonomy/task2vec ...
September 26, 2020 Yiran Luo Low Resource Machine Translation
October 03, 2020 Sheng Cheng Super-resolution

Very Good Machine Learning Classes

History: This reading group was started by TG as a weekly reading group / social meetup for PhD students stranded and locked up inside their own houses due to the COVID19 pandemic. We decided to continue it after summer ended (does it ever end in AZ?) and were too lazy to change the name. S01 of the reading group started on May 16, 2020 and ended on Oct 03, 2020. S02 resumed in May 2021, with Joshua Feinglass taking over as host.
First In-Person Meeting

Tejas Gokhale