Hi Everyone,
This document covers following points:
1. What is Teradata Control Framework?
2. Control Framework Control Patterns.
3. TERADATA Control Framework 3- Tier Architecture.
4. STREAM concept
5. TERADATA Control Framewok Components.
6. Job to start stream
7. Source to Staging Job Flow Diagram.
8. Source to Staging Job steps.
9. Staging to ODS load steps.
Teradata Control Framework
The Control Framework is a set of architectural standards, development methods, processing patterns and a code library that provides the basis for implementing a true time variant data warehouse.
•The Control Framework is an automated framework to manage the ELT processes of the Teradata Global Architecture (3 Tier EDW) to ensure accuracy and efficiency.
>The Control Framework is used in conjunction with an ELT tool – in itself it is not a tool nor a replacement for a tool.
•The Control Framework enforces audit trail and reduces programmer effort.
•The Control Framework consists of the following components.
>Process Control Data Model and standard processing columns for all cross functional model tables.
>Standard code modules for ELT processes, surrogate keys and reference code data management .
>House keeping utilities for the daily/intraday control of the ELT processes
•The Control Framework is a Teradata Professional Services Consulting Asset that is available for purchase by Teradata customers.
Control Framework Control Patterns:
•Start of Day
>Opens the business date for a given stream of ELT
•Start of Stream
>Opens the stream instance within a business date for a specific ELT batch
•End of Stream
>Completes the stream instance
•End of Day
>Completes the business date for a given stream
•Register Source File Extract
>Registers the availability of source data in the landing queue.
TERADATA CONTROL FRAMEWORK 3- TIER ARCHITECTURE
STREAM Concept
•A Stream is a collection of processes that must be completed as a whole to meaningfully transform a set of input data (files or tables) into a coherent output set of tables within the data warehouse. Each collection of dependent processes is identified by a record in this table.
•A Stream is a unit of schedulable work. Once completed a Stream may not be re-run without intervention to revert output data and CTLFW metadata to the pre-run state.
•There is no limit to the number of times a Stream may be run on any given date. Only one instance of any Stream is allowed to be run at a time.
Control Framework Components:
•Data Model that holds processing metadata
>When, what program, what source, what target, update metrics
>Static metadata, eg System, Files, Paths
>Operational metadata collected by the Standard CTLFW code at run time, e.g. metrics…
•Standard Code for the processing, control and registration patterns
>Called and executed to read from and write to the CTLFW tables.
>Guarantees
–Reliability
–Restart
–Accuracy
>Does the audit and control part of the process isolated from the individual pattern.
These lines indicates function1, function2, function3 .
Thanks & Regards,
Vishakha Nigam