About Journal Tables
So, first of all, you might ask, is what is a Journal Table? A Journal Table is a table that captures an before or after image of every change to a row in the table that is being monitored, usually with a date/time to show when the change occurred. The idea is that we be able to reconstruct a history of changes to the data in a table. Because of the overhead, you probably don't want a journal table behind all of your tables, but when you need to track who did it, when was it done, and what exactly was changed, a journal table can be a good solution. It would also help to recover from changes that shouldn't have been done. There are other solutions like Oracle's flashback query capabilities, but some databases can't do that, and some tables require a little more precise control. By the way, it is often necessary to set permissions on journal tables more stringently than permissions on the tables being journaled, so that hackers can't cover their tracks.
I personally prefer to make these before images, so that you can see what the row looked like before the change. This would imply that you need before UPDATE and before DELETE triggers that INSERT the row of the table being journaled as it exists before the change to the journal table, but no before INSERT since before the INSERT the row didn't exist. But my development lead wanted an after image, which is fine, since SQL Server only has AFTER statement triggers. Here is an example of a table to be journaled, and a journal table:
I personally prefer to make these before images, so that you can see what the row looked like before the change. This would imply that you need before UPDATE and before DELETE triggers that INSERT the row of the table being journaled as it exists before the change to the journal table, but no before INSERT since before the INSERT the row didn't exist. But my development lead wanted an after image, which is fine, since SQL Server only has AFTER statement triggers. Here is an example of a table to be journaled, and a journal table:
CREATE TABLE my_table (
my_id INTEGER,
my_char_data VARCHAR(30)
);
CREATE TABLE my_table_jn (
my_id INTEGER,
my_char_data VARCHAR(30),
operation VARCHAR(10),
date_changed DATETIME
);
If I do (on September 10):
INSERT INTO my_table (my_id, my_char_data)
VALUES (1,'Example 1');
The after INSERT trigger should do:
INSERT INTO my_table_jn (my_id, my_char_data, operation, date_changed)
VALUES (1, 'Example1','INSERT', CONVERT(datetime,'09/10/2018',101));
If I do (on September 12):
UPDATE my_table SET my_char_data = 'Example2'
WHERE my_id = 1;
The after UPDATE trigger should do:
INSERT INTO my_table_jn (my_id, my_char_data, operation, date_changed)
VALUES (1, 'Example2','UPDATE', CONVERT(datetime,'09/12/2018',101));
If I do (on September 14):
DELETE my_table
WHERE my_id = 1;
The after DELETE trigger should do:
INSERT INTO my_table_jn (my_id, my_char_data, operation, date_changed)
VALUES (1, 'Example2','DELETE', CONVERT(datetime,'09/14/2018',101));
The data for DELETE is actually a before image, since after the delete there is no data. By the way, triggers participate in the underlying transaction, so if the change to my_table is rolled back, so will the INSERT into my_table_jn.
The Script
As I said before, SDDM includes a script for adding code for Journal Tables to your DDL. Though I couldn't use Oracle's script as written, it served as an excellent starting point for my version. It also shows how you get access to the underlying SDDM data. Here is the first part of the script:
/*
Writes CREATE commands for Journal Table and Triggers for SQL Server.
variable ddlStatementsList should be used to return the list with DDL statements
that are created by script - as shown below:
ddlStatementsList.add(new java.lang.String(ddl));
other available variables:
- model - relational model instance
- pModel - physical model instance
- table - the table in relational model
- tableProxy - table definition in physical model
*/
Since the original script is written in Javascript, and mine is written in Groovy, I needed to change the syntax to Groovy, but much is the same or similar, including comments. Notice that SDDM hands you access points to the SDDM data - listed in the comments above. But it doesn't tell you how to write to SDDM's log. Fortunately, Dave Schleis provided the following code:
// get a handle to the application object
def app = oracle.dbtools.crest.swingui.ApplicationView
app.log("Creating DDL for Journal table for ${table.name}");
"ddl" is a variable to hold the code to be added to the ddl being exported for the table to be journaled. In the original, this was a string variable, but strings in Groovy are immutable. When you do "ddl = ddl + 'a string'" you are really creating a new string object. So I changed it to a StringBuilder, which in Groovy and Java is an object to which you can append more data, without the waste of discarding old strings and creating new ones.
StringBuilder ddl;
String lname;
//journal table name suffix
jnTabSuf = "_jn";
// trigger name suffix
jnAISuf = "_jn_ai";
jnAUSuf = "_jn_au";
jnADSuf = "_jn_ad";
prompt = model.appView.settings.includePromptInDDL;
useSchema = model.appView.settings.isIncludeSchemaInDDL();
if(model.storageDesign.open){
if(useSchema){
lname = tableProxy.longName;
}else{
lname = tableProxy.name;
}
}else{
if(useSchema){
lname = table.longName;
}else{
lname = table.name;
}
}
Here you will see a major advantage of using Groovy for your DDL Transformation (and other) scripts. Groovy has a GString type, similar to strings in Java and Javascript, but you can embed variables in your GStrings. In other languages you would have to concatenate strings. This is a great space and time saver when the script is really code that writes code.
if(prompt){
ddl = new StringBuilder("PRINT 'Creating Journal Table for ${lname};'\n");
}else{
ddl = new StringBuilder("");
}
app.log("Creating Journal Table DDL.");
Most of the rest of the code is appending strings (GStrings) to the ddl variable. Groovy overloads the "append()" method of StringBuilder to the "<<" operator, once again saving me a little time and space. Also notice that I'm using the triple quoted string in this section of code, which lets me use actual line feeds in place of the "\n" line feed character. I didn't do this throughout, because I didn't want to fool with the original Javascript code more than necessary.
ddl <<
"""CREATE TABLE ${lname}${jnTabSuf}
(${table.name}${jnTabSuf}_id INT IDENTITY(1,1) NOT NULL
,operation VARCHAR(10) NOT NULL
""";
cols = table.elements;
Here, I'm looping through the columns from the original table and writing the journal table with the same columns. I don't add the date_changed column, because my tables already have a last_update_date column. They also have a last_update_user_id. You could probably add code that says to add these to the journal table if they don't exist.
cols.each {
ddl <<
" ,$it.name $it.datatypeString";
if (it.mandatory){
ddl << " NOT NULL\n";
}else{
ddl << "\n";
}
}
/* Primary key is non-clustered because queries of the Journal table will
* usually be by the parent table's key.
*/
ddl <<
" ,CONSTRAINT ${table.name}${jnTabSuf}_pk\n" +
" PRIMARY KEY NONCLUSTERED (${table.name}${jnTabSuf}_id)\n" +
" );\n" +
"GO\n\n" +
/* So instead, we create a clustered index on the parent table's PK.
* No Foreign key, because we may delete rows of the parent table and still
* keep the journal of changes to the now-deleted rows, including the datetime of
* deletion.
*/
"CREATE CLUSTERED INDEX ${table.name}${jnTabSuf}_fki\n" +
" ON ${table.name}${jnTabSuf} (${table.name}_id);\n" +
"GO\n"
if(prompt){
ddl << "\nPRINT 'Creating Journal Triggers for ${lname};'\n";
}else{
ddl << "\n";
}
So far, my code has been similar to the original, but my triggers are very different - SQL Server triggers are not at all like Oracle triggers. Leave a comment, if you want me to write a post to compare the way it works in SQL Server with triggers to do it in Oracle.
app.log("Creating Journal Table After Insert trigger.");
ddl <<
"DROP TRIGGER IF EXISTS ${lname}${jnAISuf};\n" +
"GO\n" +
"CREATE TRIGGER ${table.name}${jnAISuf}\n" +
" ON ${lname}\n" +
" AFTER INSERT AS\n" +
"BEGIN \n" +
" INSERT INTO ${lname}${jnTabSuf}\n" +
" (operation\n";
cols.each {
ddl <<
" ,$it.name\n";
}
ddl <<
" )\n" +
" SELECT 'INSERT' AS operation\n"
cols.each {
ddl <<
" ,$it.name\n";
}
ddl <<
" FROM inserted;\n" +
"END;\n" +
"GO\n\n";
The rest of the code is pretty similar.
app.log("Creating Journal Table After Update trigger.");
ddl <<
"DROP TRIGGER IF EXISTS ${lname}${jnAUSuf};\n" +
"GO\n" +
"CREATE TRIGGER ${table.name}${jnAUSuf}\n" +
" ON ${lname}\n" +
" AFTER UPDATE AS\n" +
"BEGIN \n" +
" UPDATE t\n" +
" SET t.last_update_date = CURRENT_TIMESTAMP\n" +
" FROM ${lname} AS t\n" +
" INNER JOIN inserted AS i\n" +
" ON t.${table.name}_id = i.${table.name}_id;\n\n" +
" INSERT INTO ${lname}${jnTabSuf}\n" +
" (operation\n";
cols.each {
ddl <<
" ,$it.name\n";
}
ddl <<
" )\n" +
" SELECT 'UPDATE' AS operation\n"
cols.each {
ddl <<
" ,$it.name\n";
}
ddl <<
" FROM inserted;\n" +
"END;\n"+
"GO\n\n"
app.log("Creating Journal Table After Delete trigger.");
ddl <<
"DROP TRIGGER IF EXISTS ${lname}${jnADSuf};\n" +
"GO\n" +
"CREATE TRIGGER ${table.name}${jnADSuf}\n" +
" ON ${lname}\n" +
" AFTER DELETE AS\n" +
"BEGIN \n" +
" INSERT INTO ${lname}${jnTabSuf}\n" +
" (operation\n";
cols.each {
ddl <<
" ,$it.name\n";
}
ddl <<
" )\n" +
" SELECT 'DELETE' AS operation\n"
cols.each {
if (it.name == "last_update_date") {
ddl <<
" ,CURRENT_TIMESTAMP AS $it.name\n";
} else {
ddl <<
" ,$it.name\n";
}
}
ddl <<
" FROM deleted;\n" +
"END;\n"+
"GO\n"
The last step is to add the ddl variable that I've been building to the DDL that will be exported. Since my variable is a StringBuilder, not a String,we just need to use its toString method.
ddlStatementsList.add(ddl.toString());
So that's it. You are welcome to cut and paste this into your own SDDM project. Hope this was useful.
/*
Writes CREATE commands for Journal Table and Triggers for SQL Server.
variable ddlStatementsList should be used to return the list with DDL statements
that are created by script - as shown below:
ddlStatementsList.add(new java.lang.String(ddl));
other available variables:
- model - relational model instance
- pModel - physical model instance
- table - the table in relational model
- tableProxy - table definition in physical model
*/
Since the original script is written in Javascript, and mine is written in Groovy, I needed to change the syntax to Groovy, but much is the same or similar, including comments. Notice that SDDM hands you access points to the SDDM data - listed in the comments above. But it doesn't tell you how to write to SDDM's log. Fortunately, Dave Schleis provided the following code:
// get a handle to the application object
def app = oracle.dbtools.crest.swingui.ApplicationView
app.log("Creating DDL for Journal table for ${table.name}");
"ddl" is a variable to hold the code to be added to the ddl being exported for the table to be journaled. In the original, this was a string variable, but strings in Groovy are immutable. When you do "ddl = ddl + 'a string'" you are really creating a new string object. So I changed it to a StringBuilder, which in Groovy and Java is an object to which you can append more data, without the waste of discarding old strings and creating new ones.
StringBuilder ddl;
String lname;
//journal table name suffix
jnTabSuf = "_jn";
// trigger name suffix
jnAISuf = "_jn_ai";
jnAUSuf = "_jn_au";
jnADSuf = "_jn_ad";
prompt = model.appView.settings.includePromptInDDL;
useSchema = model.appView.settings.isIncludeSchemaInDDL();
if(model.storageDesign.open){
if(useSchema){
lname = tableProxy.longName;
}else{
lname = tableProxy.name;
}
}else{
if(useSchema){
lname = table.longName;
}else{
lname = table.name;
}
}
Here you will see a major advantage of using Groovy for your DDL Transformation (and other) scripts. Groovy has a GString type, similar to strings in Java and Javascript, but you can embed variables in your GStrings. In other languages you would have to concatenate strings. This is a great space and time saver when the script is really code that writes code.
if(prompt){
ddl = new StringBuilder("PRINT 'Creating Journal Table for ${lname};'\n");
}else{
ddl = new StringBuilder("");
}
app.log("Creating Journal Table DDL.");
Most of the rest of the code is appending strings (GStrings) to the ddl variable. Groovy overloads the "append()" method of StringBuilder to the "<<" operator, once again saving me a little time and space. Also notice that I'm using the triple quoted string in this section of code, which lets me use actual line feeds in place of the "\n" line feed character. I didn't do this throughout, because I didn't want to fool with the original Javascript code more than necessary.
ddl <<
"""CREATE TABLE ${lname}${jnTabSuf}
(${table.name}${jnTabSuf}_id INT IDENTITY(1,1) NOT NULL
,operation VARCHAR(10) NOT NULL
""";
cols = table.elements;
Here, I'm looping through the columns from the original table and writing the journal table with the same columns. I don't add the date_changed column, because my tables already have a last_update_date column. They also have a last_update_user_id. You could probably add code that says to add these to the journal table if they don't exist.
cols.each {
ddl <<
" ,$it.name $it.datatypeString";
if (it.mandatory){
ddl << " NOT NULL\n";
}else{
ddl << "\n";
}
}
/* Primary key is non-clustered because queries of the Journal table will
* usually be by the parent table's key.
*/
ddl <<
" ,CONSTRAINT ${table.name}${jnTabSuf}_pk\n" +
" PRIMARY KEY NONCLUSTERED (${table.name}${jnTabSuf}_id)\n" +
" );\n" +
"GO\n\n" +
/* So instead, we create a clustered index on the parent table's PK.
* No Foreign key, because we may delete rows of the parent table and still
* keep the journal of changes to the now-deleted rows, including the datetime of
* deletion.
*/
"CREATE CLUSTERED INDEX ${table.name}${jnTabSuf}_fki\n" +
" ON ${table.name}${jnTabSuf} (${table.name}_id);\n" +
"GO\n"
if(prompt){
ddl << "\nPRINT 'Creating Journal Triggers for ${lname};'\n";
}else{
ddl << "\n";
}
So far, my code has been similar to the original, but my triggers are very different - SQL Server triggers are not at all like Oracle triggers. Leave a comment, if you want me to write a post to compare the way it works in SQL Server with triggers to do it in Oracle.
app.log("Creating Journal Table After Insert trigger.");
ddl <<
"DROP TRIGGER IF EXISTS ${lname}${jnAISuf};\n" +
"GO\n" +
"CREATE TRIGGER ${table.name}${jnAISuf}\n" +
" ON ${lname}\n" +
" AFTER INSERT AS\n" +
"BEGIN \n" +
" INSERT INTO ${lname}${jnTabSuf}\n" +
" (operation\n";
cols.each {
ddl <<
" ,$it.name\n";
}
ddl <<
" )\n" +
" SELECT 'INSERT' AS operation\n"
cols.each {
ddl <<
" ,$it.name\n";
}
ddl <<
" FROM inserted;\n" +
"END;\n" +
"GO\n\n";
The rest of the code is pretty similar.
app.log("Creating Journal Table After Update trigger.");
ddl <<
"DROP TRIGGER IF EXISTS ${lname}${jnAUSuf};\n" +
"GO\n" +
"CREATE TRIGGER ${table.name}${jnAUSuf}\n" +
" ON ${lname}\n" +
" AFTER UPDATE AS\n" +
"BEGIN \n" +
" UPDATE t\n" +
" SET t.last_update_date = CURRENT_TIMESTAMP\n" +
" FROM ${lname} AS t\n" +
" INNER JOIN inserted AS i\n" +
" ON t.${table.name}_id = i.${table.name}_id;\n\n" +
" INSERT INTO ${lname}${jnTabSuf}\n" +
" (operation\n";
cols.each {
ddl <<
" ,$it.name\n";
}
ddl <<
" )\n" +
" SELECT 'UPDATE' AS operation\n"
cols.each {
ddl <<
" ,$it.name\n";
}
ddl <<
" FROM inserted;\n" +
"END;\n"+
"GO\n\n"
app.log("Creating Journal Table After Delete trigger.");
ddl <<
"DROP TRIGGER IF EXISTS ${lname}${jnADSuf};\n" +
"GO\n" +
"CREATE TRIGGER ${table.name}${jnADSuf}\n" +
" ON ${lname}\n" +
" AFTER DELETE AS\n" +
"BEGIN \n" +
" INSERT INTO ${lname}${jnTabSuf}\n" +
" (operation\n";
cols.each {
ddl <<
" ,$it.name\n";
}
ddl <<
" )\n" +
" SELECT 'DELETE' AS operation\n"
cols.each {
if (it.name == "last_update_date") {
ddl <<
" ,CURRENT_TIMESTAMP AS $it.name\n";
} else {
ddl <<
" ,$it.name\n";
}
}
ddl <<
" FROM deleted;\n" +
"END;\n"+
"GO\n"
The last step is to add the ddl variable that I've been building to the DDL that will be exported. Since my variable is a StringBuilder, not a String,we just need to use its toString method.
ddlStatementsList.add(ddl.toString());
So that's it. You are welcome to cut and paste this into your own SDDM project. Hope this was useful.
No comments:
Post a Comment