Ray Anyi Rao

Ph.D. Student, Multimedia Laboratory
The Chinese University of Hong Kong
Hong Kong, China
Email: anyirao [at]

View Anyi (Ray) Rao's profile on Google Scholar        View Anyi (Ray) Rao's codes on Github        View Anyi (Ray) Rao's profile on LinkedIn

Short Bio

Ray Anyi Rao is a Ph.D. student at Multimedia Laboratory in the Chinese University of Hong Kong, advised by Dahua Lin and Bolei Zhou. He got the B.S. from EE Department Honor Class, Nanjing University in 2018, ranking 1/183. He was a research assistant of AIM Lab, a research intern of HKU CS, and a member of LAMDA group. He served as a HUAWEI campus ambassador.


  • 2020-07: Three papers accpeted to ECCV 2020! All about movie research. They are Cinematic Style Analysis, Online Multi-modal Person Search, and a holistic dataset MovieNet (Spotlight).
  • 2020-05: Our CVPR 2020 work Local-to-Global Multi-modal Movie Scene Segmentation will appear at LUV 2020 (15-min talk) and Sight and Sound 2020 (5-min talk)
  • 2020-02: One paper is accpeted to CVPR 2020! It proposes a new temporal video segmenting method based on a new dataset. It is going to help divide a long video into semantic continuous short videos and output a structural representation. And it also provides research opportunities towards story/plot understanding in long videos with a semantic unit.
  • 2020-01: HotFlip is included in AllenNLP
  • Publication

    [1] A Unified Framework for Shot Type Classification Based on Subject Centric Lens
    Anyi Rao, Jiaze Wang, Linning Xu, Xuekun Jiang, Qingqiu Huang, Bolei Zhou, Dahua Lin
    European Conference on Computer Vision (ECCV), 2020
    [Comming Soon]

    [2] MovieNet: A Holistic Dataset for Movie Understanding
    Qingqiu Huang, Yu Xiong, Anyi Rao, Jiaze Wang, Dahua Lin
    European Conference on Computer Vision (ECCV), 2020
    [Comming Soon]

    [3] Online Multi-modal Person Search in Videos
    Jiayue Xia, Anyi Rao, Linning Xu, Qingqiu Huang, Dahua Lin
    European Conference on Computer Vision (ECCV), 2020
    [Comming Soon]

    [4] A Local-to-Global Approach to Multi-modal Movie Scene Segmentation
    Anyi Rao, Linning Xu, Yu Xiong, Guodong Xu, Qingqiu Huang, Bolei Zhou, Dahua Lin
    In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2020
    [Paper] [Webpage] [Code]

    [5] HotFlip: White-Box Adversarial Examples for Text Classification
    Javid Ebrahimi, Anyi Rao, Daniel Lowd, Dejing Dou
    In Proceedings of Annual Meeting of the Association for Computational Linguistics (ACL), 2018
    It is included in AllenNLP, an open-source NLP research library
    [Paper] [Poster] [AllenNLP] [Code]

    [6] Automatic Music Accompaniment
    Anyi Rao, Francis Lau
    In arXiv:1803.09033, 2018
    [Paper] [Presentation] [Code]

    Undergrad Research Experience

    Robust Training with Word-level Adversity for NLP
    Sept.2017-April.2018   Advanced Integration and Mining Lab (AIM), Eugene, OR, United States of America
    Advisor: Prof. Dejing Dou (Director, Head of Baidu Big Data Lab) and Prof. Daniel Lowd (Director of Graduate Studies)

  • Proposed an efficient word-level gradient-based adversarial examples generation approach for training robust models.
  • Evaluated the method with convolutional neural networks across a wide range of sentence-level classification tasks and the method using adversarial training achieved excellent performances on benchmarks.
  • This work on Adversarial examples for NLP was featured in an article in The Register
  • Automatic Music Accompaniment Using Probabilistic Machine Learning
    Jul.2017-Aug.2017   The University of Hong Kong, Hong Kong S.A.R.
    Advisor: Prof. Francis Lau (Associate Dean)

  • Proposed a fast decoding algorithm to deal with performance errors and reduce computational complexity from O(n2) to O(n). It is able to work in real time with practical length scores.
  • Constructed a comprehensive system and developed a parallel Hidden Markov Model for score following.
  • Developed a new free open-source Windows-based automatic music follower and accompanist.
  • Express Bill Handwritten Telephone Number Recognition
    Feb.2017-Jul.2017   Learning And Mining from DatA Group, Nanjing University
    Director: Prof. Zhi-Hua Zhou (ACM, AAAI, AAAS, IEEE Fellow, Academy of Europe Foreign Member)

  • Incorporated an NLP lexicon-based transcription into an RCNN-CTC network and improved its performance on standard benchmarks.
  • Accomplished 95% accuracy on each digit and 90% accuracy on a series of telephone number.
  • Real-time 3D Surface Reconstruction Using Lidar (Light Detection And Ranging)
    Aug.2016-Sept.2017   Visual Sensing and Graphics Lab (VISG Lab), Nanjing University
    Supervisor: Prof. Sidan Du (Director)

  • Proposed a novel line of sight algorithm to real-time reconstruct surface and achieved state-of-the-art results.
  • Employed a new surface lattice data structure in implicit surface update for memory efficiency.
  • Presented a real-time 3D reconstruction pipeline for large-scale Lidar point cloud.
  • Implemented parallel computation to update the implicit surface faster and Motion Estimation and Mapping to register point cloud. [Report] [Video]
  • Academic

    His GPA ranked No.1 in each semester during his undergraduate studies with overall GPA: 3.96/4.00 and Rank: 1/183.
    He finished major curricula in 2 years and learned a bunch of online courses. [Whole]

    Undergrad Course Projects

    Computer Vision 3D Human Poses Estimation from a Single Image [Presentation]
    Used sparse coding to reduce the ambiguities in the 3D pose estimation; Enforced human-portion constraints to get a minimization problem

    Convex Optimization Road Car Flow and Its Effect on Roads [Report]
    Constructed models to depict road circumstances and to predict car motion using Markov Decision Process and OD estimated matrix; Improved the models in terms of traffic load and vehicle type.

    Probability and Stochastic Process Monte Carlo Method for Multidimensional Integrals [Report]

    Machine Learning Construction of a Spam Filtering System

    Microcomputers and Interface Techniques x86 Assembly Language Programming [Report]

    Signal Processing Single Photon Detector [Report] [Presentation]
    Designed a 64-channel low-noise pre-amplifier using a symmetric structure, reduced noise by 100 times, drew an 8-layers circuit board.

    Leadership Experience

    Co-Founder of a Children Care Volunteer Program     Sep.2015-Dec.2015
    Co-founded a psychological consulting program to promote left-behind children growth and education. Volunteered to teach left-behind children Math and English in a junior high school located in the remote, underdeveloped Xiushui county. Recognized as a key team leader in the successful Warm One Hundred Campaign, which raised money for left-behind children. Our group received excellence award from China Foundation for Poverty Alleviation.

    Vice President of a Young Volunteers Association at Nanjing University     Jun.2015-Jun.2016
    Organized and participated in over 100 out-of-school and 20 in-school activities covering over 1000 volunteers. Our association received a volunteers association excellence award.


  • Hong Kong PhD Fellowship
  • 2018
  • National Scholarship awarded by the China Ministry of Education, the highest honor in China
  • 2015
  • Provincial-Level Merit Student awarded by the Jiangsu Provincial Government, the highest honor in Jiangsu Province
  • 2017
  • Nanjing University Top-Grade Scholarship, awarded to 12 students out of all undergraduate students at the university, the highest honor in the university
  • 2018
  • SenseTime Scholarship, awarded to 30 students out of all AI major undergraduate students in China
  • 2017
  • Zhenggang Scholarship, top 40 students in Nanjing University
  • 2016
  • Zhenggang Jingying Scholarship
  • 2017
  • Nanjing University People Scholarship
  • 2016, 2017
  • Nanjing University Outstanding Student Leader Award
  • 2015
  • Nanjing University Outstanding Student Award
  • 2016
  • Nanjing University Top Volunteer Excellence Award
  • 2015
  • World ranking 32nd in 2016 Calculus World Cup
  • 2016
  • Meritorious winner prize in the 2016 National Mathematical Contest in Modeling
  • 2016
  • Best paper in the 2014 University Electronics Design Contest
  • 2014
  • Gold Medal in Invitational National Mathematical Olympiad
  • 2013


    Research intern group leader in CHANGJIANG ELEC. TECH. [Media Report]
    Member of Nanjing University Student Choir (Joyful Snowflakes written by Chih-mo Hsu)
    Vice president of a Young Volunteers Association [Media Report]
    Bronze medal at Nanjing University 55th Sports Meet [Media Report]
    Language: A bit of Japanese and Cantonese, Native Mandarin, Full proficiency English