Data Engineering
Get ready to build production-grade data pipelines with Foundry's entire data engineering suite.
The Data Engineering Bootcamp will get you and your team ready to build production-grade data pipelines — fast. It’s a focused 3 days of instruction and hands-on exercises that cover the essential applications and capabilities for data engineering in Foundry. We also cover pipeline management, optimization, security, and general best practices.
By the end of the bootcamp, your team will be ready to:
- Integrate & transform new datasources
- Analyze Develop, deploy, and maintain large-scale data pipelines using both code and no-code tooling data
- Create object types in the ontology backed by data pipelines up alerts for abnormal fluctuations in data
- Protect and govern data access in order to comply with organizational rules or governmental regulations
- Troubleshoot and optimize data pipelines the provenance of datasets and analytical deliverables
Hands-on exercises
Throughout the bootcamp, attendees will have plenty of chances to practice what they learn. We schedule time for Q&A and discussions that can address your organization’s specific needs and circumstances.
Who is it for?
The Data Engineering Bootcamp is for current data engineers who are new to Palantir Foundry. Attendees should be comfortable with SQL and Python. Familiarity with Spark is a plus.
We are happy to offer extended bootcamps if your team prefers more detailed instructions on any topic. This bootcamp can be customized for teams who do not know how to code, and who are interested in learning to build production-ready data pipelines using Foundry’s no-code data engineering application, Pipeline Builder.
Your organization must be a current Palantir customer as trainings will take place on your Foundry stack.
What does it contain?
The bootcamp schedule spans two and a half days and includes the following modules.
{{curriculum}}
Data Connection
- Setting up data sources and syncs
- Streaming data connections
- Webhooks and data export
Building data pipelines with SparkSQL or PySpark
- Branching code and datasets
- Code Authoring application
- Building a batch pipeline
- Building an incremental pipeline
Deploying and scheduling data pipelines
- Data Lineage application
- Using Contour for ad hoc pipeline output validation
- Pipeline deployment best practices
- Pipeline organization & documentation
- Chapter
Pipeline monitoring
- Data Health & Data Expectations
- Automatic issues filing
- Managing resource usage and costs
Frequently-asked questions
Pipeline Builder is the most recent addition to Foundry’s data integration suite. It lets anyone create production-grade data pipelines without needing to know how to code.
With Pipeline Builder, you have the ability to develop and deploy batch, incremental and even streaming pipelines in a point-and-click interface. You benefit from the same version control of logic and data that Foundry’s code-based tools pioneered, and Pipeline Builder is fully integrated into Foundry’s pipeline orchestration, monitoring and alerting, and security capabilities. Pipelines made in Pipeline Builder can even be exported to code if needed.
And if you want to learn to code data pipelines, Pipeline Builder is a good way to start: it relies on the same git-based version control workflow as Foundry’s Code Authoring application; it will let you improve your understanding of Spark best practices; and it will let you gain experience at structuring and organizing the logic of data pipelines. Imagine trying to learn all that while simultaneously learning your first programming language.
Pipeline Builder is the most recent addition to Foundry’s data integration suite. It lets anyone create production-grade data pipelines without needing to know how to code.
With Pipeline Builder, you have the ability to develop and deploy batch, incremental and even streaming pipelines in a point-and-click interface. You benefit from the same version control of logic and data that Foundry’s code-based tools pioneered, and Pipeline Builder is fully integrated into Foundry’s pipeline orchestration, monitoring and alerting, and security capabilities. Pipelines made in Pipeline Builder can even be exported to code if needed.
And if you want to learn to code data pipelines, Pipeline Builder is a good way to start: it relies on the same git-based version control workflow as Foundry’s Code Authoring application; it will let you improve your understanding of Spark best practices; and it will let you gain experience at structuring and organizing the logic of data pipelines. Imagine trying to learn all that while simultaneously learning your first programming language.
Pipeline Builder is the most recent addition to Foundry’s data integration suite. It lets anyone create production-grade data pipelines without needing to know how to code.
With Pipeline Builder, you have the ability to develop and deploy batch, incremental and even streaming pipelines in a point-and-click interface. You benefit from the same version control of logic and data that Foundry’s code-based tools pioneered, and Pipeline Builder is fully integrated into Foundry’s pipeline orchestration, monitoring and alerting, and security capabilities. Pipelines made in Pipeline Builder can even be exported to code if needed.
And if you want to learn to code data pipelines, Pipeline Builder is a good way to start: it relies on the same git-based version control workflow as Foundry’s Code Authoring application; it will let you improve your understanding of Spark best practices; and it will let you gain experience at structuring and organizing the logic of data pipelines. Imagine trying to learn all that while simultaneously learning your first programming language.
Meet your teacher
Taylor helps Palantir customers & partners get more out of Foundry
Previously, Taylor worked at Palantir deploying Foundry to commercial and government customers around the world. His experience includes working with customers in healthcare, transportation, media, finance, and manufacturing.
