Architecture

The capture function is performed by capture background process cnnn (where nnn is a capture process number, for example c001). Capture process components are: reader, preparer, and builder servers. Some of the possible capture process states are:

- CAPTURING CHANGES: process scans the redo logs for changes that evaluate to TRUE against the capture process rule sets

- PAUSED FOR FLOW CONTROL: there is a delay while LCR’s are enqueued

- CREATING LCR: redo log entries are converted to LCR

Propagation is using job queue background process (jnnn) to propagate messages from the source to the destination queue.

The apply components are coordinator (corresponding background process is annn) that gets transactions from the apply process reader and passes them to apply servers. The apply process uses parallel execution servers (pnnn).

Some of the apply states are:

- SPILLING: unapplied messages are spilled from memory to hard disk

- DEQUEUE MESSAGES: dequeuing messages

Configuration

The single PL/SQL procedure – DBMS_STREAMS_ADM.MAINTAIN_SCHEMAS compleetely sets up basic schema level replication. Certain tasks must be completed, though, before this procedure runs in step 8.

1. Configure source database to run in ARCHIVELOG mode

2. Set up Streams administrator STRMADMIN on both databases (source and target)

3. Adjust initialization parameters on both databases for Streams replication

4. Create database link from source (ORCLA.WORLD) to target (ORCLB.WORLD) database

5. Create database link from target (ORCLB.WORLD) to source (ORCLA.WORLD) database

(This link is needed because I will use the network Data Pump export/import which is not using an export dump file; standard export/import using dump file is also possible )

6. (Recommended) Create directory on source machine (It will contain generated replication script.)

7.( Recommended) Run DBMS_STREAMS_ADM.MAINTAIN_SCHEMAS with perform_actions parameter set to FALSE on source database to generate the replication script. This is a dry run to verify the script’s correctness and avoid time-consuming replication cleanup (in case the replication setup script fails.)

8. Run DBMS_STREAMS_ADM.MAINTAIN_SCHEMAS procedure on the source database with perform_actions parameter set to TRUE. This will startt the replication setup.

9. Verify that DDL and DML changes on ORCLA.WORLD are propagated to ORCLB.WORLD

Steps in detail

For steps 1 – 4, please refer to the Mishra’s article mentioned above. They are identical.

5. Create a database link on target database:

connect strmadmin/strmadmin@ORCLB.world

CREATE DATABASE LINK ORCLA.WORLD

CONNECT TO STRMADMIN

IDENTIFIED BY STRMADMIN

USING ‘ORCLA.WORLD’;

6. Create a directory on the source database where the script generated by dbms_streams_adm.maintain_schema will be stored:

CONNECT strmadmin/strmadmin@orcla.world

CREATE OR REPLACE DIRECTORY ADMIN AS ‘/home/oracle/Streams’;

7. Generate script schema_replication.sql in Oracle admin directory

(/home/oracle/Streams on Linux):

CONN strmadmin/strmadmin@orcla.world

BEGIN

DBMS_STREAMS_ADM.MAINTAIN_SCHEMAS(

schema_names => ‘scott‘,

source_database => ‘orcla.world‘,

destination_database => ‘orclb.world‘,

capture_name => ‘capture_scott‘,

capture_queue_table => ‘rep_capture_queue_table‘,

capture_queue_name => ‘rep_capture_queue‘,

capture_queue_user => null,

apply_name => ‘apply_scott‘,

apply_queue_table => ‘rep_dest_queue_table‘,

apply_queue_name => ‘rep_dest_queue‘,

apply_queue_user => null,

propagation_name => ‘prop_scott‘,

log_file => ‘exp.log‘,

bi_directional => false,

include_ddl => true,

instantiation => dbms_streams_adm.instantiation_schema_network,

perform_actions => false,

script_name => ‘schema_replication.sql‘,

script_directory_object => ‘admin’

);

END;

(A complete description of parameters used is listed in Appendix.)

The /home/oracle/Streams/schema_replication.sql script now contains commands to completely configure SCOTT schema replication. A brief description of scripts contents follows:

- Supplemental logging data is added for all tables in the SCOTT schema; it is required to log additional data to redo logs if you want Streams to work properly

- DBMS_STREAMS_ADM.SET_UP_QUEUE procedure is run to create capture queue and capture table

- DBMS_STREAMS_ADM.ADD_SCHEMA_PROPAGATION_RULES is run to add rules to the positive rule set for propagation; this is queue-to-queue propagation

- Propagation is temporarily disabled

- DBMS_STREAMS_ADM.ADD_SCHEMA_RULES is run to add rules to capture process

- Datapump schema mode import (network) is run from ORCLB.WORLD database ( SCOTT schema is exported from ORCLA.WORLD and imported into ORCLB.WORLD; DBMS_DATAPUMP package is used for this purpose)

- Capture process is started on ORCLA.WORLD database

- Apply queue REP_DEST_QUEUE on the target database is configured using the DBMS_STREAMS_ADM.SET_UP_QUEUE procedure

- Schema rules for the apply process are added (DBMS_STREAMS_ADM.ADD_SCHEMA_RULES procedure is executed )

- A tag is added to apply process and apply process is started

- Propagation is enabled

8. We will now execute MAINTAIN_SCHEMAS procedure; perform_actions parameter is changed to TRUE.

BEGIN

DBMS_STREAMS_ADM.MAINTAIN_SCHEMAS(

schema_names => ‘scott‘,

source_directory_object => NULL,

destination_directory_object => NULL,

source_database => ‘orcla.world‘,

destination_database => ‘orclb.world‘,

capture_name => ‘capture_scott‘,

capture_queue_table => ‘rep_capture_queue_table‘,

capture_queue_name => ‘rep_capture_queue‘,

capture_queue_user => null,

apply_name => ‘apply_scott‘,

apply_queue_table => ‘rep_dest_queue_table‘,

apply_queue_name => ‘rep_dest_queue‘,

apply_queue_user => null,

propagation_name => ‘prop_scott‘,

log_file => ‘exp.log‘,

bi_directional => false,

include_ddl => true,

instantiation => dbms_streams_adm.instantiation_schema_network,

perform_actions => true

);

END;

/

The execution of this procedure will take some time, depending on the schema size, number of objects, volume of data etc. When it completes all DML and DDL changes to tables in the SCOTT schema on ORCLA.WORLD, the database will be propagated to ORCLB.WORLD database.

Customization

I will now customize replication setup to exclude EMP table from replication (Metalink note 239623.1).

<!–[if !supportLists]–>1) <!–[endif]–>Find capture rule name for schema:

CONN strmadmin/strmadmin@orcla.world

select r.rule_name, r.rule_owner

from dba_rule_set_rules rs, dba_capture c , dba_rules r

where c.rule_set_name = rs.rule_set_name

and c.rule_set_owner = rs.rule_set_owner

and rs.rule_name = r.rule_name

and rs.rule_owner = r.rule_owner

and upper(r.rule_condition) like ‘%:DML%’;

RULE_NAME RULE_OWNER

—————– ——————————<

SCOTT15 STRMADMIN

set long 100000

select rule_condition

from dba_rules

where rule_name = ‘SCOTT15′;

RULE_CONDITION

———————————————————–

((((:dml.get_object_owner()

= ‘SCOTT’) and :dml.get_source_database_name() = ‘ORCLA.WORLD’ )) and

(:dml.get_compatible() <= dbms_streams.compatible_10_2))

2) Stop capture process on ORCLA.WORLD ( similar to Advanced Replication quiesce

operation ) to allow for rule change:

begin

dbms_capture_adm.stop_capture(‘capture_scott’);

end;

/

3) Alter capture rule to exclude EMP table from replication

begin

dbms_rule_adm.alter_rule(‘SCOTT15′,

‘(:dml.get_object_owner() = ”SCOTT” and not ‘ ||

‘:dml.get_object_name() = ”EMP” and ‘ ||

‘:dml.is_null_tag() = ”Y” )’);

end;

/

4) Start the capture process:

begin

dbms_capture_adm.start_capture(‘capture_scott’);

end;

/

Changes to emp table are now not propagated.

Monitoring

Replication is now fully configured. DML and DDL changes are flowing from source to target database.How do we find what is happening with capture, propagate and apply processes ?

To find out the state of the capture process:

select state from v$streams_capture;

STATE

———————————

CAPTURING CHANGES

To find out total number of messages propagated on sending side:

select total_msgs from v$propagation_sender

TOTAL_MSGS

——————–

15863766

To find out what is the state of apply reader:

select state from v$streams_apply_reader;

STATE

————–

SPILLING

Conclusion

Oracle Streams is relatively new tool primarily designed to help move real time data between databases.

The easy setup, possibility of light data transformation, and good performance make it welcome addition to a DBA’s toolset.

LISTING/Definitions

DBMS_STREAMS_ADM.MAINTAIN_SCHEMAS parameter description.

schema_names is the name of schema that will be propagated ( scott in this example ).

perform_actions is the parameter that specifies if a script should be actually executed. If the value is TRUE, then procedure actions will be performed. If it is set to FALSE, then the procedure will only generate a script.

capture_name is the name of capture process configured to capture changes made on source database.

capture_queue_table is the name of the queue table for each queue used by a capture process.

capture_queue_name is the name of each queue used by a capture process.

capture_queue_user is set to NULL, meaning procedure will not grant any privileges.

propagation_name is the name of each propagation configured to propagate changes.

log_file is the name of the Data Pump export log file.

bi_directional is set to FALSE because we are creating one-way replication from the source to the target database.

include_ddl is TRUE, which means both DML and DDL changes will be replicated from the source to the target database.

instantiation is set to DBMS_STREAMS_ADM.INSTANTIATION_SCHEMA. It means that target schema instantiation will be done using Data Pump export/import dump file.

perform_actions is set to FALSE. Procedure is generating a script that can be reviewed before the actual run happens.

script_name is set to schema_replication.sql. This script will be written to Oracle directory object admin. It will contain all steps to configure replication environment.

script_directory_object is set to admin. This is the name of Oracle directory object.