February 2, 2021 / Last updated : March 20, 2023 irfan AAAI

Paper in AAAI 2021 on “Semantic MapNet: Building Allocentric Semantic Maps and Representations from Egocentric Views”

We study the task of semantic mapping – specifically, an embodied agent (a robot or an egocentric AI assistant) is given a tour of a new environment and asked to build an allocentric top-down semantic map (‘what is where?’) from egocentric observations of an RGB-D camera with known pose (via localization sensors). Importantly, our goal is to build neural episodic memories and spatio-semantic representations of 3D spaces that enable the agent to easily learn subsequent tasks in the same space – navigating to objects seen during the tour (‘Find chair’) or answering questions about the space (‘How many chairs did you see in the house?’).

October 28, 2020 / Last updated : March 20, 2023 irfan UIST

Paper in ACM UIST 2020 on “Automatic Video Creation From a Web Page”

Creating marketing videos from scratch can be challenging, especially when designing for multiple platforms with different viewing criteria. We present URL2Video, an automatic approach that converts a web page into a short video given temporal and visual constraints. URL2Video captures quality materials and design styles extracted from a web page, including fonts, colors, and layouts. Using constraint programming, URL2Video’s design engine organizes the visual assets into a sequence of shots and renders to a video with a user-specified aspect ratio and duration. Creators can review the video composition, modify constraints, and generate video variation through a user interface. We learned the design process from designers and compared our automatically generated results with their creation through interviews and an online survey. The evaluation shows that URL2Video effectively extracted design elements from a web page and supported designers by bootstrapping the video creation process.
Overview of the paper
September 1, 2020 / Last updated : March 25, 2023 irfan ISWC

Paper in ISWC 2020 on “Masked reconstruction based self-supervision for human activity recognition”

The ubiquitous availability of wearable sensing devices has rendered large scale collection of movement data a straightforward endeavor. Yet, annotation of these data remains a challenge and as such, publicly available datasets for human activity recognition (HAR) are typically limited in size as well as in variability, which constrains HAR model training and effectiveness. We introduce ..
August 25, 2020 / Last updated : March 20, 2023 irfan ECCV

Paper in ECCV 2020 on “Neural Design Network: Graphic Layout Generation with Constraints”

Graphic design is essential for visual communication with layouts being fundamental to composing attractive designs. Layout generation differs from pixel-level image synthesis and is unique in terms of the requirement of mutual relations among the desired components. We propose a method for design layout generation that can satisfy user-specified constraints.

April 27, 2020 / Last updated : March 20, 2023 irfan ICLR

Paper in ICLR 2020 on “Decentralized Distributed PPO: Solving PointGoal Navigation”

We present Decentralized Distributed Proximal Policy Optimization (DD-PPO), a method for distributed reinforcement learning in resource-intensive simulated environments. DD-PPO is distributed (uses multiple machines), decentralized (lacks a centralized server), and synchronous (no computation is ever ‘stale’), making it conceptually simple and easy to implement.

March 6, 2020 / Last updated : March 3, 2021 irfan Presentations

Keynote Speaker at CNS/AANS Spine Summit 2020, Las Vegas, Nevada, March 6, 2020, on the topic of “Data-driven Innovation”

Honored to be invited as the keynote/guest speaker at the Spine Summit 2020, The 36th Annual Meeting of the American Association of Neurological Surgeons (AANS) and the Congress of Neurological Surgeons (CNS) on March 6, 2020, at the Cosmopolitan of Las Vegas, in Las Vegas, Nevada, USA. Here is the full program of the conference. […]
November 2, 2019 / Last updated : February 21, 2021 irfan ICCV

Paper in ICCV Workshop on Geometry Meets Deep Learning Workshop on “Floors are Flat: Leveraging Semantics for Real-Time Surface Normal Prediction”

Abstract We propose 4 insights that help to significantly improve the performance of deep learning models that predict surface normals and semantic labels from a single RGB image. These insights are: (1) denoise the ”ground truth” surface normals in the training set to ensure consistency with the semantic labels; (2) concurrently train on a mix […]