Abstract We introduce the task of scene-aware dialog. Our goal is to generate a complete and natural response to a question about a scene, given video and audio of the scene and the history of previous turns in the dialog. To answer successfully, agents must ground concepts from the question in the video while leveraging […]
A nice write-up in Georgia Tech’s Research Horizons Magazine about ML@GT Machine learning has been around for decades, but the advent of big data and more powerful computers has increased its impact significantly — moving machine learning beyond pattern recognition and natural language processing into a broad array of scientific disciplines. A subcategory of artificial intelligence, […]
A short write-up on one of my recent publications. A new login authentication approach could improve the security of current biometric techniques that rely on video or images of users’ faces. Known as Real-Time Captcha, the technique uses a unique challenge that’s easy for humans — but difficult for attackers who may be using machine learning […]
A TEDx talk that I recently did. In this talk, the speaker takes you on a journey of how AI systems have evolved over time. DIRECTOR OF MACHINE LEARNING AT GEORGIA INSTITUTE OF TECHNOLOGY Dr. Irfan Essa is a professor in the school of Interactive Computing and the inaugural Director of Machine Learning at Georgia […]
Paper Abstract Purpose: Basic surgical skills of suturing and knot tying are an essential part of medical training. Having an automated system for surgical skills assessment could help save experts time and improve training efficiency. There have been some recent attempts at automated surgical skills assessment using either video analysis or acceleration data. In this […]
Paper Abstract Carefully managing the presentation of self via technology is a core practice on all modern social media platforms. Recently, selfies have emerged as a new, pervasive genre of identity performance. In many ways unique, selfies bring us full circle to Goffman—blending the online and offline selves together. In this paper, we take an […]
Paper U. Ahsan, M. D. Choudhury, and Irfan Essa (2017), “Towards Using Visual Attributes to Infer Image Sentiment Of Social Events,” in Proceedings of The International Joint Conference on Neural Networks, Anchorage, Alaska, US, 2017. [PDF] [BIBTEX] @InProceedings{ 2017-Ahsan-TUVAIISSE, address = {Anchorage, Alaska, US}, author = {Unaiza Ahsan and Munmun De Choudhury and Irfan Essa}, […]
Paper Abstract We propose to leverage concept-level representations for complex event recognition in photographs given limited training examples. We introduce a novel framework to discover event concept attributes from the web and use that to extract semantic features from images and classify them into social event categories with few training examples. Discovered concepts include a variety of objects, scenes, […]
Machine Learning at Georgia Tech Seminar Series Speaker: Irfan Essa Date/Time: March 1, 2017, 12n Abstract The Interdisciplinary Research Center (IRC) for Machine Learning at Georgia Tech (ML@GT) was established in Summer 2016 to foster research and academic activities in and around the discipline of Machine Learning. This center aims to create a community that […]
Paper A. Zia, D. Castro, and I. Essa (2016), “Fine-tuning Deep Architectures for Surgical Tool Detection,” in Workshop and Challenges on Modeling and Monitoring of Computer Assisted Interventions (M2CAI), Held in Conjunction with International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI), Athens, Greece, 2016. [PDF] [WEBSITE] [BIBTEX] @InProceedings{ 2016-Zia-FDASTD, address = {Athens, […]