Mark Boss

Mark Boss

Research Scientist

Stability AI

Biography

Mark Boss is a researcher at Stability AI. He worked at Unity Technologies before and completed his PhD at the University of Tübingen in the computer graphics group of Prof. Hendrik Lensch. His research interests lie at the intersection of machine learning and computer graphics, focusing mainly on inferring physical properties (shape, material, illumination) from images.

If you are interested in a research collaboration, please drop me a    mail with your CV.

Interests
  • Machine Learning
  • Computer Graphics
  • Inverse Rendering
Education

  • PhD in Computer Science, 2023

    University of Tübingen

  • MSc in Computer Science, 2018

    University of Tübingen

  • BSc in Computer Science, 2016

    Osnabrück University of Applied Sciences

News

SF3D: Stable Fast 3D Mesh Reconstruction with UV-unwrapping and Illumination Disentanglement
01 Aug 2024 - ArXiv submission and Project page online. It also became a full product at Stability AI.
Two Papers Accepted at ECCV 2024
01 Jul 2024 - Collaborative Control for Geometry-Conditioned PBR Image Generation and SV3D: Novel Multi-view Synthesis and 3D Generation from a Single Image using Latent Video Diffusion got accepted for ECCV 2024.
SV3D: Novel Multi-view Synthesis and 3D Generation from a Single Image using Latent Video Diffusion
15 Mar 2024 - ArXiv submission and Project page online
Collaborative Control for Geometry-Conditioned PBR Image Generation
09 Feb 2024 - ArXiv submission and Project page online
SHINOBI: Shape and Illumination using Neural Object Decomposition via BRDF Optimization In-the-wild
19 Jan 2024 - ArXiv submission and Project page online
See all

Recent Posts

NeRF at CVPR 2023
It is now my third time writing a summary of NeRFy things at a conference. This time it is the big one: CVPR. The list of accepted papers is massive again, with 2359 papers.
May 1, 2023 49 min read NeRF, Literature Review
NeRF at ECCV 2022
I recently went through the the provisional programm of ECCV 2022. After my last post on “NeRF at NeurIPS” got such great feedback, and I anyway compiled a list of all NeRFy things, I decided to do it all again.
Oct 1, 2022 12 min read NeRF, Literature Review
NeRF at NeurIPS 2022
Inspired by Frank Dellaert and his excellent series on the original NeRF Explosion and the following ICCV/CVPR conference gatherings, I decided to look into creating a NeurIPS 22 rundown myself.
Last updated on Sep 30, 2022 8 min read NeRF, Literature Review

Recent & Upcoming Talks

Inverse Rendering for Games
Asset production in the game industry is time-consuming, and since “The Vanishing of Ethan Carter” photogrammetry has …
Nov 28, 2023 4:40 PM — 5:15 PM Hochschule der Medien, Stuttgart
Slides Video
Neural Reflectance Decomposition
In this talk, I will present our recent work on decomposing an object into its shape, reflectance, and illumination. This highly …
Jul 26, 2022 10:00 AM — 11:00 AM Adobe Research
Slides

Publications

SF3D: Stable Fast 3D Mesh Reconstruction with UV-unwrapping and Illumination Disentanglement
We present SF3D, a novel method for rapid and high-quality textured mesh reconstruction from a single image. Utilizing the Large …
Cite PDF Website Video Code Model Demo
SF3D: Stable Fast 3D Mesh Reconstruction with UV-unwrapping and Illumination Disentanglement
SV3D: Novel Multi-view Synthesis and 3D Generation from a Single Image using Latent Video Diffusion
We present Stable Video 3D (SV3D) - a latent video diffusion model for high-resolution, image-to-multi-view generation of orbital …
Cite PDF Website Video
SV3D: Novel Multi-view Synthesis and 3D Generation from a Single Image using Latent Video Diffusion
Collaborative Control for Geometry-Conditioned PBR Image Generation
Current 3D content generation builds on generative models that output RGB images. Modern graphics pipelines, however, require …
Cite PDF Website
Collaborative Control for Geometry-Conditioned PBR Image Generation
SHINOBI: Shape and Illumination using Neural Object Decomposition via BRDF Optimization In-the-wild
We present SHINOBI, an end-to-end framework for the reconstruction of shape, material, and illumination from object images captured …
Cite PDF Video Website
SHINOBI: Shape and Illumination using Neural Object Decomposition via BRDF Optimization In-the-wild
Neural Reflectance Decomposition
Creating relightable objects from images or collections is a fundamental challenge in computer vision and graphics. This problem is …
Cite Slides Publication
Neural Reflectance Decomposition
SAMURAI: Shape And Material from Unconstrained Real-world Arbitrary Image collections
Inverse rendering of an object under entirely unknown capture conditions is a fundamental challenge in computer vision and graphics. …
Cite Slides arXiv Video Code Scenes
SAMURAI: Shape And Material from Unconstrained Real-world Arbitrary Image collections
An open-source smartphone app for the quantitative evaluation of thin-layer chromatographic analyses in medicine quality screening
Substandard and falsified medicines present a serious threat to public health. Simple, low-cost screening tools are important in the …
Cite Project PDF Code Website
An open-source smartphone app for the quantitative evaluation of thin-layer chromatographic analyses in medicine quality screening
Neural-PIL: Neural Pre-Integrated Lighting for Reflectance Decomposition
Decomposing a scene into its shape, reflectance and illumination is a fundamental problem in computer vision and graphics. Neural …
Cite arXiv Video Code
Neural-PIL: Neural Pre-Integrated Lighting for Reflectance Decomposition
NeRD: Neural Reflectance Decomposition from Image Collections
Decomposing a scene into its shape, reflectance, and illumination is a challenging but important problem in computer vision and …
Cite arXiv Video Code Dataset
NeRD: Neural Reflectance Decomposition from Image Collections
Two-shot Spatially-varying BRDF and Shape Estimation
Capturing the shape and spatially-varying appearance (SVBRDF) of an object from images is a challenging task that has applications in …
Cite Paper + Supplementary arXiv Video Code Dataset
Two-shot Spatially-varying BRDF and Shape Estimation
Single Image BRDF Parameter Estimation with a Conditional Adversarial Network
Creating plausible surfaces is an essential component in achieving a high degree of realism in rendering. To relieve artists, who …
PDF Cite
Single Image BRDF Parameter Estimation with a Conditional Adversarial Network
Deep Dual Loss BRDF Parameter Estimation
Surface parameter estimation is an essential field in computer games and movies. An exact representation of a real-world surface allows …
PDF Cite DOI
Deep Dual Loss BRDF Parameter Estimation

Projects

Nerfherder.xyz
The amount of NeRF paper each day can become overwhelming. After several summaries, I’ve created a small tool that can create blog posts automatically but also can create daily slide decks from arXiv. The tool runs entirely independently and categorizes cs.CV, writes a summary, extracts a teaser image, and builds a slide deck.
Nerfherder.xyz
TLCyzer
TLCyzer is a free and open source smartphone app for the quantitative evaluation of thin-layer chromatographic analyses in medicine quality screening, described and validated in a scientific study.
TLCyzer
farrow-and-ball
A set of British paint manufacturer Farrow & Ball inspired color maps for matplotlib. The maps are sorted in categories for different plotting needs (Divergent, Spectral, Base Colors, Misc).
farrow-and-ball
Online BRDF Visualization
The bidirectional reflectance distribution function (BRDF) is a function that defines how incoming light is reflected outward. This web-based visualizer uses the Cook-Torrance model and shows how the material looks on a sphere, and simultaneously displays its evaluated BRDF. Especially the different reflective behavior of specular and diffuse materials are easily visible.
Online BRDF Visualization
Infomark
A free, scalable, modern and open source solution for programming lectures supporting auto-testing/grading of programming assignments scaling to thousands of students and several courses.
Infomark

Experience

 
 
 
 
 
Stability AI
Research Scientist
Stability AI
Jan 2024 – Present Germany
Research on object acquisition, usage of deep priors to reduce ambiguities, and generative AI.
 
 
 
 
 
Unity Technologies
Senior Research Scientist
Unity Technologies
Sep 2022 – Jan 2024 Germany
Research on object acquisition and generative AI.
 
 
 
 
 
University of Tübingen
Ph.D. Student
University of Tübingen
Jun 2018 – Jun 2022 Tübingen, Germany
Research on deep learning based material acquisition.
 
 
 
 
 
Google
Student Researcher
Google
Jun 2021 – Apr 2022 Germany (Remote)
Research on novel techniques for material, geometry and illumination disentanglement.
 
 
 
 
 
NVIDIA
Research Intern
NVIDIA
Apr 2019 – Jul 2019 Westford, MA
Research on casual shape and material acquisition, which resulted in the publication: “Two-shot Spatially-varying BRDF and Shape Estimation”
 
 
 
 
 
zahlz
Android Developer
zahlz
Jul 2015 – Jun 2017 Osnabrück
Development of an Android Application for a mobile payment system.