Research Engineer / Research Scientist, Vision-Language Model Pretraining
Epsilon Health · San Francisco
About Us
Epsilon Health is transforming medical imaging and diagnostics by bringing frontier AI into clinical practice. We're tackling some of healthcare's most critical challenges, building technology that directly improves patient outcomes.
This year, we've served over 300,000 patients and now serve 3,000+ daily, working with some of the largest medical institutions in the U.S. By year-end, we're on track to interpret 1% of all daily U.S. X-rays.
Our research team brings experience from DeepMind and Meta, powered by one of the industry's most comprehensive proprietary medical imaging datasets, continuously enriched by real-world clinical data. Together, we're building a future where frontier AI helps save millions of lives today, not tomorrow.
Recently out of stealth, we're proudly backed by Jack Altman, Uncork Capital, Renegade Partners, and AlleyCorp, one of NYC's most active healthcare venture firms.
About the Role
We're looking for an exceptional Research Scientist to push the frontier of vision-language models for medical imaging.
You'll own the pretraining of our multimodal foundation models, developing architectures and training methods that enable AI to understand complex medical images and generate clinically accurate radiology reports.
You'll tackle fundamental research challenges in multimodal learning, high-resolution visual understanding, and large-scale pretraining across X-rays, CT, and MRI.
Unlike traditional research labs, your work will draw on continuously growing real-world clinical data and help power models deployed into clinical practice, creating a direct feedback loop between research and patient care.
You'll work alongside a small team of researchers and engineers, with the freedom to pursue ambitious ideas and the resources to bring them to life. You'll collaborate closely with researchers responsible for post-training and reinforcement learning.
What You'll Do
Advance the frontier of multimodal AI. Design, pretrain, and scale vision-language foundation models for medical imaging.
Invent new architectures. Explore novel approaches to vision-language fusion, high-resolution image understanding, and efficient visual token representations.
Push the limits of pretraining. Develop multimodal training objectives, data mixtures, and scaling strategies across large-scale image-text datasets.
Solve visual grounding. Improve how models connect medical findings to specific regions, structures, and abnormalities within images.
Own research end to end. Take ideas from hypothesis and experimentation through large-scale distributed training, evaluation, and handoff to post-training.
Define how progress is measured. Develop rigorous evaluations for visual understanding, grounding, generalization, and downstream model performance.
Contribute to the research community. Publish original research, share technical insights, and help shape the direction of medical AI.
What We're Looking For
We care more about what you've built and discovered than where you've worked or how many years you've spent doing it.
Deep expertise in vision-language models, multimodal learning, or large-scale foundation model pretraining.
Demonstrated ability to develop, train, or improve state-of-the-art machine learning models.
Strong understanding of modern VLM architectures, multimodal training objectives, and vision-language alignment.
Experience with large-scale distributed training using PyTorch or JAX.
Ability to independently formulate research questions, design experiments, and turn promising ideas into working models.
Strong engineering skills and a track record of implementing and extending ideas from research.
A rigorous, experimental mindset and the ambition to solve difficult, unsolved problems.
Nice to Have
Publications at leading ML or computer vision conferences such as NeurIPS, ICML, ICLR, CVPR, or MICCAI.
Experience developing or scaling models such as LLaVA, Qwen-VL, InternVL, or similar multimodal architectures.
Experience training vision encoders or developing novel vision-language connector architectures.
Familiarity with high-resolution imaging, visual grounding, or 3D/volumetric data.
Experience with contrastive learning, generative pretraining, or large-scale multimodal data curation.
Background in medical imaging, radiology, or other scientific imaging applications.
Experience evaluating model factuality, interpretability, or reliability in high-stakes settings.
We welcome exceptional researchers from both academic and industry backgrounds. A traditional career path or prior healthcare experience is not required.
More jobs at Epsilon Health · Quant Research jobs in San Francisco