The changing of light patterns requires some JavaScript and trigonometric acumen on the developers’ part, but the client merely needs to reload the webrtc-perception interface to get updated JavaScript code, and tweaks to server processing code are invisible to the client device. 2. cs1290tas@lists.brown.edu—your s… Cheating & Academic Dishonesty: Do your own work. Also, put up a “safety” submission on Canvas with what you currently have, an hour prior to the deadline. Unsupervised Deep Learning for Computational Photography and Imaging Self2Self: Self-Supervised Image Denoising Self2Self with dropout: Learning self-supervised denoising from … This project also leans on another library named aiortc to implement Python-based interaction with connecting clients via WebRTC and perform useful computation on images and other data gathered through use. Save your images that you’ll use for the results and your report in png format. Applicants should hold a 4-year bachelor's degree (or equivalent). The goal of this homework is to explore the focus properties of images captured by your Tegra device. Nick Antipa*, Grace Kuo*, Ren Ng, and Laura Waller. Machine Learning . We will then use this as a basis to explore recent topics in computational photography such as motion/defocus deblurring cameras, light field cameras, and computational illumination. Lunch Break. The client device, thanks to other MediaStream features, also permits the server to detect and choose which photography settings are important for that particular camera track (such as exposure time, ISO, white balance, focus distance, rear torch status, etc). Grading: Homeworks 1 through 7 are each graded Pass/Fail. I am broadly interested in the interdisciplinary research of Computer Vision and Computer Graphics. Several applications and products already leverage WebRTC for video conferencing, gaming, media sharing, and other social applications, so it has benefited from steady growth and support since its introduction at the 2013 Google I/O developers conference. by 11:59pm on the due date. This gives you an idea of what an end-to-end system could look like, but without the rtc-shapeshifter- or rtc-deflectometry-specific details. Participants are encouraged to … Here's my GitHub. Who may apply? During my time spent in Northwestern University’s Computational Photography Lab, I divided my attention between the mothballed handheld 3D scanner project and another project oriented around WebRTC. Computational photography combines ideas in computer vision, computer graphics, and image processing to overcome limitations in image quality such as resolution, dynamic range, and defocus/motion blur. This iteration of the class makes use of material from the classes by James Tompkin, Ioannis Gkioulekas, Marc Pollefeys, and Alyosha Efros. Computational photography is the convergence of computer graphics, computer vision, optics, and imaging. Academic dishonesty will be dealt with as laid out in the student handbook. Computational Photography and Image Manipulation. My personal website. Computational Photography and Image Manipulation as a class is tought in many institutions with varying flavors. It offers a powerful tool to combine algorithms and sensing systems to outperform traditional sensors. Lectures will also be recorded for those who cannot attend during scheduled class times. Computational illumination is used within the movie industry to render the performances of live actors into digital environments. You can sign up for the page at that link using the sign-up code 6624. It is a fairly tight schedule I was a research intern at MSRA, supervised by Dr. Xun Guo. My research interests lie at the intersection of optics, computer vision, and computer graphics. 1.3 Elements of Computational Photography Traditional film-like photography involves (a) a lens, (b) a 2D planar sensor and (c) a processor that converts sensed values into an image. Announcements and discussions will take place on CampusWire. to ensure we cover many different topics. Face Morphing. His work originally used DSLR cameras to get preliminary results and he switched to using an iPhone (with some special hardware) in its final form, which made it an interesting candidate for extension through webrtc-perception. # Computational Photography (ICCP), 2014 IEEE International Conference on # # hL and hH are the one dimenionsal filters designed by our optimization # bL and bH are the corresponding Chebysheve polynomials # t is the 3x3 McClellan transform matrix # directL and directH are the direct forms of the 2d filters Aug/2020: I successfully defended my M.S. I did not provide the MATLAB scripts for these projects publicly (github, etc.) During my time spent in Northwestern University’s Computational Photography Lab, I divided my attention between the mothballed handheld 3D scanner project and another project oriented around WebRTC.WebRTC (RTC stands for Real-Time Communications) is a suite of APIs that enables the capture and transfer of video and audio content entirely through a web browser. Computational Optical Sensing and Imaging. We will provide a Nvidia Tegra tablet for each student in the course. Welcome to Winston’s homepage! Our results and a description of the work was featured in Optics Express Vol. Students will write programs that run on the phone to capture photos. I started looking at WebRTC APIs in mid-2018 to determine if our lab could use such a technology as the basis for a new scientific data collection system. ELEC_ENG 395, 495: Computational Photography Seminar “guest lecturer“, Northwestern University, 2020. To teach the fundamentals of modern camera architectures and give students hand-on experience acquiring, characterizing, and manipulating data captured using a modern camera platform. The next sections outline the goals of rtc-shapeshifter and rtc-deflectometry and how my colleagues are using webrtc-perception to achieve those goals. At present, two applications are featured in the metapackage: rtc-shapeshifter and rtc-deflectometry. William Spies is an aspiring Roboticist and Research Scientist currently serving in the Computational Photography Lab at Northwestern University. See CANVAS for the link to invite your to create your Github repository for the assignments. I am an Assistant Professor in the EECS Department at Northwestern University. The most imposing limitation was that the end system cannot require users to download a separate application, and instead ONLY use what would be available in modern web browsers. The Computational Photography Lab is led by Prof. Oliver Cossairt, Associate Professor in the Department of Electrical Engineering and Computer Science at Northwestern University. Project coming soon. getUserMedia() and other MediaStream components simplify connecting to a client device. Research. thesis! Mar/2020 We will then continue to explore more advanced topics in computer vision. We plan to stick closely to these grading guidelines, but some exceptions may be made for partial credit (e.g. Computational Optical Sensing and Imaging. Office Hours: Thursday 3-5PM - write an email to florian.willomitzer@northwestern.edu to book a 10min slot. This is a prediction of what will be covered in each week but the schedule is since these projects are still used as homework assignments for the course. He can control various photography settings remotely, trigger image capture from the rear-facing camera (with the LED light enabled), clip on his polarizer, and automate processing and results generation…and see his results while capturing data. My research interests lie in Computer Vision, Deep Learning and Computational Photography. degree in Software Engineering at Sichuan University in 2019, supervised by Prof. Jiancheng Lv.I’ve also attended the Summer workshop at National University of Singapore in Big Data & Cloud Computing with full scholarship. So if you pass all seven assignments you get an A, if you fail one assignment you get a B, if you fail two you get a C, and so on. Jeremy Lainé has put together a very useful package and I highly recommend giving it a closer look. Tao Yue. rtc-shapeshifter is a WebRTC-based tool that expands upon a concept originally presented by Chia-Kai Yeh called Shape by Shifting. Specifically, I am interested in Vision and Language, 3D vision, Neural Rendering, Computational photography, Image & Video Understanding, AR/VR & Embodied AI. Students with a bachelor’s degree in a field other than CS are encouraged to apply, but to succeed in graduate-level CS courses, they must have prerequisite coursework or commensurate experience in object-oriented programming, data structures, algorithms, linear algebra, and statistics/probability. I gave a guest lecture to the CP Seminar course. (5) Northwestern Neuroimaging and Applied Computational Anatomy (Lei Wang) 20 min (6) Michigan Institute for Data Science (Ivo Dinov), 20 min 12:00-1:00. Each homework consists of a coding and a technical writeup. If serious problem regarding an assignment arise, I am available for zoom session on an individual basis. The project “metapackage” is named webrtc-perception and is hosted over on GitHub. The Nvidia Tegra Shield is an Android-based tablet that features a 5-megapixel camera with an easy to use camera API. You can resubmit up to three homework assignments that you received a failing grade for. Optical Society of America, 2017. For coding questions that involve your own code, please make a private thread that is only visibile to TA/Instructor. Late Policy: If EITHER there is nothing on Canvas OR your code has not been pushed to by 11:59pm on the due date, you fail the assignment. Florian’s application uses webrtc-perception to access the front-facing camera on a device and change camera settings for the connected client. Its role is to overcome the limitations of traditional cameras, by combining imaging and computation to enable new and enhanced ways of capturing, representing, and … I did my bachelors at Nanjing Agricultural University. Office Hours: Thursday 3-5PM - write an email to oliver.cossairt (a) northwestern.edu to book a 10min slot. Computational Photography SIGGRAPH Course (Raskar & Tumblin), Computational Camera and Photography (Raskar, MIT), Digital and Computational Photography (Durand & Freeman, MIT), Computational Photography (Levoy & Wilburn, Stanford), Computational Photography (Belhumeur, Columbia), Computational Photography (Essa, Georgia Tech), Introduction to Visual Computing (Kutulakos, U of Toronto). When and Where to Submit Assignments: A latex writeup report for each assignment must be submitted on Canvas Florian Willomitzer The Lytro Camera captures a 4D light field of a scene, enabling photographs to be digitally refocused after images are captured. Enrollment is limited to 30 students. However, a requirement for a zoom session is to have an active Campuswire thread. The design of webrtc-perception includes a capture website, a dedicated server for processing image data, and a results display website. Furthermore, my system needed to work without requiring my colleagues to possess special hardware or be familiar with the nuances of browser APIs or web development. These glass tiles were part of a sample set from the Kokomo Opalescent Glass Works in Indiana, famous for having supplied glass to Louis Comfort Tiffany. Implementing PMD techniques on consumer devices using webrtc-perception is an alternate way to measure the surface shape by instead “scanning” the glass with the mobile device. I'm an assistant professor at Graduate School and Faculty of Information Science and Electrical Engineering, Kyushu University. Optical Society of America, 2017. Computational imaging stands in the crossroad of computer graphics, computer vision, and optics and sensors. This course is the first in a two-part series that explores the emerging new field of Computational Photography. My Ph.D. thesis was closely related to the tasks which involve moving objects present in videos or images captured from different view-points. In addition, the photography may involve (d) external illumination from point sources (e.g. Unconference Breakout Sessions (4 consecutive slots of 30-min each). 28, Issue 7. Finally, there are some details below the webrtc-perception metapackage description that talks about some specific applications for this technology, both of which have unique implications for scientific study of artistic works. subject to change as the course progresses. Image Classification. Some developers and researchers have also used WebRTC to facilitate IoT applications, serve as the framework for hobbyist projects, and have integrated it into cutting-edge computer science and robotics research. : July/2020: Starting in September I will be joining Dr. Vladlen Koltun's Intelligent Systems Lab at Intel as Research Scientist resident. (4 consecutive slots of … I received ME and PhD from Nara Institute of Science and Technology (NAIST) in 2016 and 2019, respectively. Much of my research is about Deep Learning and Camera Pipeline. Before joining Northwestern, I spent one year (Oct. 2011 – Aug. 2012) as a Postdoctoral Researcher at Columbia University, under the … Penalties include failing the class and can be more severe than that. Programming Assignment 1 This assignment is intended to familiarize you with image filtering and frequency representations. Since joining the lab, under the guidance of Dr. Oliver Cossairt and Dr. Florian Willomitzer, he has been focusing on two practical applications of computer vision for scientific data collection. The most recent code on github at 11:59pm on the due date is the code we will grade. Computational Photography. I'm interested in computational photography, computer vision and machine learning. CampusWire will be staffed at specific times, when a member of the team will be answering questions (existing and new). Soham Ghormade. studio lights). I am actively working with deep neural networks for videos and image sequences. Many of the course materials are modified from the excellent class notes of similar courses offered in other schools by Shree Nayar, Marc Levoy, Jinwei Gu, Fredo Durand, and others. Application Integration . Hi there, My name is Wang, Zi-Hao (王子豪) and I go by Winston. Each application is connected to specific active research projects in the Computational Photography Lab. Again, for a monocular method, depth from defocus (DfD) requires a comparison image. EECS 211 and/or 230 or permission from instructor. If you have a question about whether something may be considered cheating, ask, prior to submitting your work. Research. Conferences: ICCP 2011, ICCP 2010, ICCP 2009, SIGGRAPH, SIGGRAPH Asia, CVPR, ICCV, ECCV, .. A good approach is to continually check in and push to GitHub as you work. WebRTC (RTC stands for Real-Time Communications) is a suite of APIs that enables the capture and transfer of video and audio content entirely through a web browser. Canny Edge Detection Analytics cookies. You can resubmit up to three homework assignments that you received a failing grade for. Seam Carving and Lightfield Camera. Tuesdays and Thursdays 1:00pm-2:20pm CT Special thanks to the NU Computational Photography Lab for the screenshot of Kai’s work currently serving as the project thumbnail. Aug/2020: One paper accepted at SIBGRAPI 2020! Email / GitHub / LinkedIn. This course will consist of six homework assignments and no midterm or final exam. \The Role of Niche Signals in Self-Organization in Society" Teaching Instructor of Record Computing Essentials for Social Scientists - Northwestern University - Summer 2018 Social Dynamics - University of Michigan - Fall 2013 Graduate Student Instructor My … The camera parameter could be aperture, exposure, focus, film speed or viewpoint. "3D DiffuserCam: Single-Shot Compressive Lensless Imaging." Computational photography combines plentiful low-cost computing, digital sensors, actuators, and lights to escape the limitations of traditional film-like methods. I'm broadly interested in 3D-related computer vision research, including reconstruction, depth sensing, novel view synthesis, inverse graphics, computational photography, etc. This course is the first in a two-part series that explores the emerging new field of Computational Photography. I am currently taking the course CS101c: Computational Cameras with Prof. Katie Bouman. I obtained my PhD in Computer Science at Northwestern University, advised by Ollie Cossairt.My research interests include computer vision and machine learning. For example, students will learn how to estimate scene depth from a sequence of captured images. Web Application Waldo. CS331 lecture: All lectures will held live on zoom and linked through canvas. Since WebRTC is used for capture and transport, users have to rely on other resources to complete their application, such as a dedicated server to handle image and data processing tasks and return useful results. At other times, please pull together as a class and help each other, and we'll help soon. The client signals to the server when it is ready to begin data capture, and the server responds with a signal to start “measuring” with the device. Your coding must be correct, and your writeup must be clearly written (see latex template here: ) in order to receive a passing grade. Computer Vision . Colorizing. I obtained my Ph.D. in computer science from Northwestern University, where I worked on computational photography and computer vision with Oliver Cossairt in Computational Photography Lab . rtc-deflectometry was demonstrated on the Kokomo sample glass tiles, on decorative pieces we acquired for measurement purposes, and on various other objects (even those not strictly made of glass) that exhibit specular reflection. Sep/2020: I started working at Dr. Vladlen Koltun's Intelligent Systems Lab at Intel. webrtc-perception uses the WebRTC framework to establish a connection between a server and a client device in a seamless manner. Homework is due and assigned on the dates below. https://www.sciencemag.org/news/2019/02/new-app-reveals-hidden-landscapes-within-georgia-o-keeffe-s-paintings, https://www.mccormick.northwestern.edu/news/articles/2019/02/diagnosing-art-acne-in-georgia-okeeffe-paintings.html, featured in Optics Express Vol. Computational Photography & Computer Vision Image Warping and Mosaicing. About Projects Resume Contact. The server handles gathering data from the client and performs application-specific computation on all the gathered data. 1:00-3:00. We use analytics cookies to understand how you use our websites so we can make them better, e.g. Pieces commissioned by Tiffany usually bear artistic and historical relevance, but traditional surface measurement systems can be difficult to situate and leverage if the glass work is installed and immobile. However, utilising a technique from computational photography called coded aperture, we can obtain absolute depth using just a single image.The idea in coded aperture is similar … Email / Google Scholar / Github flash units) and area sources (e.g. (5) Northwestern Neuroimaging and Applied Computational Anatomy (Lei Wang) 20 min (6) Michigan Institute for Data Science (Ivo Dinov), 20 min 12:15‐1:15 Lunch Break 1:15‐3:15 Unconference Breakouts Informal self‐organized sessions (30‐minutes each), round‐robin rotations. This did threaten to constrain the potential capabilities somewhat, but also ensured a broader potential audience and subsequent use. Robotics:AI Techniques . If you are interested, please contact the instructor to discuss! I even got to do a bit of hand modeling for the feature’s preview image! Florian Schiffers These sample tiles have a particular surface shape that, if accurately captured, can be attributed to Kokomo’s specific roller table process. When paired with some JavaScript I wrote for generating sinusoidal patterns on the K1’s display, he can generate any number of periodic image patterns on the display, use WebRTC to record image captures of the morphed pattern, transmit them to the processing server, and see the phase mapping results in real-time. PMD, for the unfamiliar, can be described as projecting light in varying structured patterns and using a camera element to perceive how a surface affects the reflection of the pattern. Our work was presented at 2019’s AAAS conference and highlighted by AAAS on Science magazine’s website, as well as featured on Northwestern University’s Engineering News reel. New methods offer unbounded dynamic range and variable focus, lighting, viewpoint, resolution and depth of … This course will first cover the fundamentals of image sensing and modern cameras. I am a Master student studying Computer Science at Northwestern University, IL, advised by Prof. Oliver Cossairt.I received my B.Eng. The server does all this through the use of Python and aiortc to connect with a client via WebRTC without needing to use a web browser itself. I am currently a third-year master student at Beihang University, where I work on computational photography under the supervision of Prof. Feng Lu. Our lab also looked at this project as a chance to create a system that could eventually be used by individuals outside of our laboratory, namely art curators and conservators, for historical or scientific documentation purposes. My aim was to develop an image capture framework that could be immediately usable for multiple ongoing research projects. The most recent submission in Canvas at that point, is the one we grade. Computational Social Science research luncheons, Northwestern University. While I will not go into deep technical detail on his work, I included some slides from a presentation we held for one of the university’s scientific interest groups on October 19th, 2018: In short, Kai has been using the webrtc-perception framework to make it easier for him to recover surface normal maps with an off-the-shelf NVIDIA SHIELD K1 tablet though the use of photometric stereo measurement. This also confers some advantages, as operators can improve the processing code on the fly, change camera controls and presentation details on the respective websites, and fix issues without users needing to download or install any new files or update applications. 1. I am now a computer vision engineer at Apple. Filters and Frequencies. Email / GitHub / Google Scholar / LinkedIn. Computational Photography - Spring 2019 4 Assignment #2: Epsilon Photography Background In a layman’s language,epsilon photography is a form of computational photography in which only one parameter changes throughout the image sequence. Here are three projects I implemented for the Computer Vision and Computational Photography course I took Fall 2015 at the University of Pennsylvania. In particular, Dr. Florian Willomitzer, the leading CPL post-doctoral researcher, was eager to measure some special glass tiles that we had in the lab. Future Video Synthesis with Object Motion Prediction Yue Wu, Rongrong Gao, Jaesik Park, Qifeng Chen CVPR, 2020 Paper / arXiv Code. Artificial Intelligence & Computational Photography - Haoban. CampusWire—your first stop for questions and clarifications. Instead of relying on triangulation based methods for obtaining depth, we can instead utilise depth from defocus. The featured implementations attempt to do this as close to real-time as possible, so that the user in control of the measurement client can evaluate the measurement process in a sort of feedback loop. We will be checking for code duplication. A-/B+, etc.). Associate Professor Nanjing University School of Electronic Science and Technology Computational Sensing and Imaging Lab E-Mail: yuetao@nju.edu.cn Tao Yue received his B.S. 28, Issue 7 in March 2020, and there is even a patent pending on this particular combined integration of PMD and mobile devices. My research interest stems from my deep fascination with upcyling. GitHub is where people build software. For each assignment that you fail, your grade gets lowered by one letter. Mail: florian.schiffers (a) northwestern.edu Computational Photography . Students should have experience with Python programming. Oliver Cossairt Examples of application-specific code is contained within the “content” folder, while the metapackage itself serves as the issue tracker and documentation holder for all contained content. rtc-deflectometry is a WebRTC-based tool that implements Phase Measuring Deflectometry (PMD) in order to optically measure surfaces that exhibit specular reflection. I am interested in Image Processing, Computational Photography and Computer Vision. The device used for data capture was again an NVIDIA SHIELD K1 tablet. Personal portfolio powered by Jekyll and GitHub Pages . Computational photography combines ideas in computer vision, computer graphics, and image processing to overcome limitations in image quality such as resolution, dynamic range, and defocus/motion blur. EECS 395/495: Introduction to Computational Photography . More than 50 million people use GitHub to discover, fork, and contribute to over 100 million projects. they're used to gather information about the pages you visit and how many clicks you need to accomplish a task. A barebones illustration of the webrtc-perception framework is shown in the following figure. The instructors are extremely thankful to the researchers for making their notes available online. This includes free response answers and code. Office hours are replaced with increased Campuswire activity on myside. The Python code converts the results of the computation into a format which can be transmitted to another, separate website designed to display (and make available, if necessary) the results. Your code must be pushed to your individual GitHub Classroom code repository, also at 11:59pm on the due date. Recent Projects. This system has made it far easier to perform surface measurements of painted works of art for the purposes of preservation and restoration. Disclaimer.