**Johan Nysjö's GFX website** Here you can find some of my spare time graphics programming projects, as well as software and publications from my earlier work as a PhD student in medical 3D image analysis and visualization. I am particularly interested in volume rendering, ray tracing, global illumination, postprocessing effects, and problems in the borderland between image analysis and rendering. # Graphics programming ## PTVol ![ ](images/ptvol_chameleon.png width="700px") PTVol is a GPU-accelated path tracer that uses delta tracking and physically-based rendering to produce near-photorealistic visualizations of volumetric image datasets, for instance, 3D computed tomography (CT) scans. [Read more here](https://bitbucket.org/johannysjo/ptvol/). ## IsoPath ![ ](images/isopath_dragon_rescaled.png width="700px") IsoPath is another GPU-accelerated path tracer that can be used for generating isosurface visualizations of volumetric images or voxelized meshes. Uses traditional ray marching instead of delta tracking. [Read more here](https://bitbucket.org/johannysjo/ptvol_isosurf/). ## PBRVol ![ ](images/pbrvol.png width="700px") PBRVol is a real-time volume renderer that combines ray marching, physically-based shading, volumetric ambient occlusion, deep shadow maps, and HDR lighting to produce realistic renderings of volumetric images such as 3D CT or MR scans. [Read more here](https://bitbucket.org/johannysjo/pbrvol/). ## Screen-space subsurface scattering ![ ](images/skin_shading.png width="700px") A deferred renderer that implements screen-space subsurface scattering (SSS, or SSSSS). It can be used for producing realistic renderings of translucent materials such as skin, vax, or marble, where some of the incoming light penetrates the surface and is scattered and attenuated by the subsurface medium. [Read more here](https://bitbucket.org/johannysjo/skin_shading/). ## Order-independent transparency (OIT) zoo ![ ](images/oit.png width="700px") I wanted to learn more about transparency rendering and, in particular, order-independent transparency (OIT) and ended up implementing a small zoo of different OIT rendering techniques during a summer vacation :) [Read more here](https://bitbucket.org/johannysjo/oit). ## Simple PT ![ ](images/simple_pt_bunnies.png width="700px") My (somewhat neglected) pet path tracer. It supports rendering of analytic shapes (spheres or boxes) and instanced triangle meshes, with emissive objects or HDR environment maps as light sources. The renderer runs on the CPU and has no acceleration structures except bounding boxes, so rendering meshes is... really slow. But it works. [Read more here](https://bitbucket.org/johannysjo/simple_pt/). ## HLCR - Happy Little Cloud Renderer ![ ](images/happy_little_clouds.png width="700px") A CPU-based volume renderer that can be used for rendering voxelized meshes (e.g., bunnies) as happy little clouds :) Imported models are "cloudified" by offsetting the voxel coordinates with 3D fractional Brownian motion (fBm) noise. The idea was to implement both a traditional ray marching backend and a more modern delta tracking-based backend with progressive refinement, but currently the renderer only supports ray marching. [Read more here](https://bitbucket.org/johannysjo/hlcr/). ## Shadertoy demos ![ ](images/shadertoy_demos.png width="700px") I really enjoy writing shader code and have a few public shadertoy demos. You can find them here: [https://www.shadertoy.com/user/myrkott](https://www.shadertoy.com/user/myrkott). # Software ## BoneSplit ![ ](images/bonesplit.png width="700px") BoneSplit is an interactive 3D painting tool for segmenting bones and bone fragments in computed tomography (CT) volume images. It combines efficient graph-based segmentation algorithms with isosurface volume rendering and interactive 3D painting to enable users to rapidly mark and segment bones of interest. Typically, an accurate segmentation result can be obtained within only a few minutes, compared with up to several hours for more conventional manual segmentation tools. The segmented bones can be exported as either meshes (STL models) or labeled volumes (VTK files), which can then be imported into other software and be used for, e.g., 3D printing, virtual surgery planning, research studies, or training data for machine learning models. Supports import of DICOM and VTK image files. See my WSCG 2015 paper for more details. Note that the current version of the segmentation algorithm is based on the Image Foresting Transform (IFT) by Falcao et al., whereas the original segmentation algorithm presented in the paper was based on GPU-accelerated Random Walks. The software is free to use for research and non-commercial purposes. Please read the license file below for more information. Also, if you publish results obtained with BoneSplit, I would appreciate if you cite this website and the references listed in the BibTeX file below. [Download .zip (124 MB)](https://drive.google.com/uc?export=download&id=1qVB_TypBOU35lwxPFMYF6NXbNYMWzgLn) | [Demo video (46 MB)](https://drive.google.com/uc?export=download&id=16iKqevpalYO7q2EuXXlycSjIOAfhPaMo) | [Installation](bonesplit/installation.txt) | [License](bonesplit/LICENSE.txt) | [BibTeX](bonesplit/references.txt) ## OrbSeg ![ ](images/orbseg.png width="700px") OrbSeg is a semi-automatic tool for segmenting the orbits (eye-sockets) in 3D computed tomography (CT) images. It implements a volumetric sculpting brush that allows the user to quickly mark and fill out the orbit. The anterior limit of the orbit is defined by a smooth surface that closely follows the shape and contour of the orbital rim. Since the segmentation method does not incorporate any prior shape information, it can be used for segmentation of both intact and fractured orbits, as well as malformed orbits or other types of cranial cavities (e.g., the sinuses or the intracranial volume). Supports import of DICOM and VTK image files. The software is free to use for research and non-commercial purposes. The paper describing the software and underlying segmentation method is not published yet (a short summary can be found in my [PhD thesis](http://www.diva-portal.org/smash/record.jsf?pid=diva2:954103)), but please contact me if you want to download a trial version of the software and try it out on your data. [Demo video (34.9 MB)](https://drive.google.com/uc?export=download&id=1SdKNEsx9Uxk0AjX66TtGzH4XZkD997Vd) # Publications This is a selection of publications from my work as a PhD student in medical image analysis and visualization at the Centre for Image Analysis, Uppsala University, 2011-2016. A more complete and up-to-date list of publications can be found on my Google Scholar profile. There is also some information on my old university web page and ResearchGate profile. I am no longer working at the university or actively involved in the research there, but feel free to contact me if you have any questions about the publications or related methods or software. ## PhD thesis - ![ ](images/thesis_cover.png width="128px") Johan Nysjö. Interactive 3D Image Analysis for Cranio-Maxillofacial Surgery Planning and Orthopedic Applications. Uppsala University, 2016. ## Journal papers - J. Nilsson, J. Nysjö, A.P. Carlsson, A. Thor. Comparison analysis of orbital shape and volume in unilateral fractured orbits. Journal of Cranio-Maxillofacial Surgery Vol.46, No.3, pp. 381-387, 2018. - A. Christersson, J. Nysjö, L. Berglund, F. Malmberg, I. Sintorn, I. Nyström, S. Larsson. Comparison of 2D radiography and a semi-automatic CT-based 3D method for measuring change in dorsal angulation over time in distal radius fractures. Skeletal Radiology, 2016. - Johan Nysjö, Filip Malmberg, Ida-Maria Sintorn, Ingela Nyström. BoneSplit - A 3D Texture Painting Tool for Interactive Bone Separation in CT Images. Journal of WSCG, Vol.23, No.2, pp. 157-166, 2015. - A. Hast, J. Nysjö, A. Marchetti. Optimal RANSAC - Towards a Repeatable Algorithm for Finding the Optimal Set. Journal of WSCG, Vol.21, No.1, pp. 21-30, 2013. - R.H. Khonsari, M. Friess, J. Nysjö, G. Odri, F. Malmberg, I. Nyström, E. Messo, J.M. Hirsch, E.A.M. Cabanis, K.H. Kunzelmann, J.M. Salagnac, P. Corre, A. Ohazama, P.T. Sharpe, P. Charlier and R. Olszewski. Shape and Volume of Craniofacial Cavities in Intentional Skull Deformations. American Journal of Physical Anthropology, Volume 151, Issue 1, pp. 110-119, May 2013. ## Conference papers - Johan Nysjö, Albert Christersson, Ida-Maria Sintorn, Ingela Nyström, Sune Larsson, Filip Malmberg. Precise 3D Angle Measurements in CT Wrist Images. In Proceedings of International Conference on Image Analysis and Processing (ICIAP), Naples, Italy, 2013. - Johan Nysjö, Albert Christersson, Filip Malmberg, Ida-Maria Sintorn, Ingela Nyström. Towards User-Guided Quantitative Evaluation of Wrist Fractures in CT Images. In Proceedings of International Conference on Computer Vision and Graphics (ICCVG), Warsaw, Poland, 2012. - Lennart Svensson, Johan Nysjö, Anders Brun, Ingela Nyström, Ida-Maria Sintorn. Rigid Template Registration in MET Images Using CUDA. In Proceedings of International Conference on Computer Vision Theory and Applications (VISAPP), Rome, Italy, 2012. - Ingela Nyström, Johan Nysjö, Filip Malmberg. Visualization and Haptics for Interactive Medical Image Analysis: Image Segmentation in Cranio-Maxillofacial Surgery Planning. In Proceedings of the 2nd International Visual Informatics Conference (IVIC), Selangor, Malaysia, 2011. ## Other - I. Nyström, P. Olsson, J. Nysjö, F. Nysjö, F. Malmberg, S. Seipel, J. Hirch, I. B. Carlbom. Virtual Cranio-Maxillofacial Surgery Planning with Stereo Graphics and Haptics. Computer-Assisted Musculoskeletal Surgery - Thinking and Executing in 3D, 2016. Book chapter. - Johan Nysjö. Orbit Segmentation for Cranio-Maxillofacial Surgery Planning. MSc thesis, Uppsala University, 2011. # CV Working experience: - May 2022-now: Rendering programmer at Ubisoft Stockholm. - 2017-2022: Senior system developer at RaySearch Laboratories AB. I worked as an image analysis specialist in the Geometry team and as a graphics programmer in the VTK teams, focusing on developing and maintaining visualization and image analysis tools for radiation treatment planning based on 3D CT and MR scans. Some of my core tasks were to develop a parametric model of the eye that could be used for planning proton beam treatment of eye tumors, and to develop new rendering components in OpenGL and Vulkan for visualizing geometric models of organs and tumors as well as different types of volumetric image data. - 2017 (4 months): Researcher at Uppsala University Hospital. Developed methods for analysing cranial volumes in 3D CT scans of craniosynostosis patients. - 2011-2016: PhD student in medical image analysis and visualization at the Centre for Image Analysis, Uppsala University. - 2011 (3 months): Programmer in the ProViz project at the Centre for Image Analysis, Uppsala University. Worked during the summer on developing methods and tools for analysing and visualizing proteins in 3D electron tomography images. Education: - 2011-2016: PhD student in medical image analysis and visualization at the Centre for Image Analysis, Uppsala University. - 2005-2011: MSc student in computer science at Uppsala University. I took courses in, e.g., image analysis, computer graphics, scientific visualization, high-performance computing, and machine learning, and did my master thesis work at the Centre for Image Analysis. Skills: - Graphics programming, scientific visualization, medical image analysis - C++, Python, C# - OpenGL, GLSL, compute shaders, Vulkan (basics), a bit of OpenCL and CUDA - 3D printing # Contact