LINCC Frameworks

Program Overview

Working with a team of professional software engineers and data scientists at two sites, we support the LSST community through the development of cloud-based analysis frameworks for LSST science.

We seek to enable the delivery of key computational infrastructure and code for petabyte-scale analyses, mechanisms to search for one-in-a-million events in continuous streams of data, and community organizations and communication channels that enable researchers to develop and share their algorithms and software. Our systems work on a range of environments, from commercial cloud to HPC, supporting the wide range of computing resources used by the astronomical community.

LINCC Frameworks engages the LSST science community in prioritizing and designing the software frameworks needed to support early science. The five-year development timeline for LINCC Frameworks has a phased approach for deliverables, to interface with increasingly large and more complex datasets, and engaging with an increasingly broader community. 

For more information, contact Andy Connolly and Rachel Mandelbaum. Please select LINCC Frameworks on the contact form. You can also sign up for our mailing list to receive notices related to LINCC Frameworks, such as announcements of Tech Talks.

Areas of Focus & Software

The LINCC Frameworks team has identified several areas where the team can make transformative software contributions to the Rubin science community. Our current areas of focus have been driven by the needs of the Rubin community, and our existing software projects address many of these areas. We will continue to identify new opportunities for collaboration, to coordinate with groups already working in these areas, and to seek other areas where software infrastructure development could strongly impact community software development for LSST. Read more about the scientific use cases and some of the known technical challenges on this page. For a more comprehensive description of technical projects, including technical documents, visit the team’s wiki page. 

Community Engagement and Collaboration

To encourage the science community to help shape the design and functionality of the analysis system, we will deliver:

Software Frameworks and General Purpose Tools

Frameworks will run on both commercial cloud and HPC that support scaling cross-cutting scientific analysis to Rubin scale data. We’ll also make contributions to general-purpose tools, including existing open-source projects and cross-project infrastructure.

Incubators

With $20K research grants, the LINCC Frameworks Incubator Program brings research teams together with the LINCC Frameworks software team to develop their own science analyses using the LINCC Frameworks tools and framework. Check out our call for proposals and our incubator awardees.

Workshops and Hackathons

The LINCC Frameworks team will host a series of workshops and hackathons, in partnership with the Rubin Observatory and LSST Science Collaborations to train the the Rubin community to make use of LINCC Frameworks tools. More information about upcoming events can be found on our Workshops and Hackathons webpage.

LSST-DA has previously hosted workshops to identify the functionality and priorities for the software frameworks and the tools LINCC Frameworks have built. The first LINCC Frameworks community engagement activity was a workshop in 2022 called From Data to Software to Science with the Rubin Observatory LSST. We compiled a Rubin LSST YouTube playlist that provides an introduction to Rubin Observatory and what the LSST data products will look like.  We also contributed to a community white paper on cross-cutting analysis software needs that will support a wide range of early LSST science cases. 

Tech Talks

LINCC Frameworks Tech Talks is a virtual forum to connect the Rubin LSST software community, showcasing work by the broad Rubin software and archives community designed to enable LSST science. Talks are delivered monthly on every second Thursday at 1pm ET/ 10am PT / 7pm CET, via Zoom (zoom link). You can see these and other events on our public calendar.

LINCC Frameworks Office Hours

The LINCC Frameworks team holds weekly Office Hours to help the community learn more about our software tools and how to use them for their own analysis. The primary goal will be to answer questions about or provide hands-on support for the LINCC Frameworks software, such as HATS, LSDB, nested-pandas, RAIL (a DESC project to which LINCC Frameworks contributes), KBMOD, TDAstro, FIBAD, or the python project template. If availability allows, we are also happy to discuss general software engineering questions.

LINCC Frameworks Office Hours will be held virtually at 1pm ET / 10am PT / 7pm CET on most Thursdays via Zoom (zoom link). The second Thursday of every month is the LINCC Frameworks Tech Talk series will be held (in place of office hours). You can see these and other events on our public calendar.

All are welcome at the LINCC Frameworks Office Hours. We encourage people who need help using LINCC Frameworks software or are just interested in finding out more to attend LINCC Frameworks Office Hours.

LINCC Frameworks Team

The LINCC Frameworks team is composed of research scientists, software engineers, and associated faculty across multiple institutions as well as leadership from LSST-DA. The research and engineering team spans two primary locations, Seattle (University of Washington) and Pittsburgh (Carnegie Mellon University and the University of Pittsburgh), with roughly half the team in each location. Each location hosts two or more research scientists and four to five engineers with local tech leads and a central engineering director. The team is dedicated to providing an inclusive environment for all people and abides by a code of conduct.