How To Group Records in a Data Package (DTP) in SAP Netweaver BW (BI)

Posted by on Oct 6, 2010 in SAP Business Intelligence | 10 comments

How To Group Records in a Data Package (DTP) in SAP Netweaver BW (BI)

Here is an example where I wanted to specify how the records are read and formed when data is loaded in BW. That is, I wanted to ensure certain groups of data are logically packaged in one data package (DTP) and not sprinkled across two different data packages.

In this example, I have created a custom data model that shows the related serial number(s) for a given delivery number, item, and material number. When loading the infocube, I want to ensure that all serial numbers for a given delivery number, item, and material are contained within the same data transfer package (DTP).

However, the settings for Semantic Groups are unavailable by default (as seen below)

Fig. 1 Semantic Group is disabled by default.

Before you can specify the Semantic Grouping, you will need to decide how error records are handled.

Navigate to the ‘Update’ tab and click on the ‘Error Handling’ window to see the four options:

  • Deactivated
  • No Update, No Reporting (default)
  • Valid Records Update, No Reporting (Request Red)
  • Valid Records Update, Reporting Possible (Request Green)

To learn more about error handling, please visit the following page. http://help.sap.com/saphelp_nw70/helpdata/en/42/fbd598481e1a61e10000000a422035/content.htm


Error Stack Setting in SAP Netweaver BW (BI)

Fig. 2 Default Error Stack Setting

In my scenario, I want to be alerted when errors occur and I want process the errors before the records are sent to the data target.

Selecting the option “Valid Records Update, No Reporting (Request Red)” will leave the error records in the error stack and set the QM status for this request to Red. Now that I have chosen my error handing option, I can set Semantic Group.

Error Stack Setting in SAP Netweaver BW (BI)

Fig 3. Custom Error Package Setting

Navigate to the ‘Extraction’ tab and confirm that a Green checkmark appears next to the ‘Semantic Group’ button. This indicates that a selection has been made.

Click on the ‘Semantic Group’ button and confirm that a ‘Selection of Key Fields for Error Stack’ window appears.

Now, you can determine how the system groups the records when reading from the data source. Unselect the unnecessary fields. Leave the key fields for your logical group as checked.

Semantic Group via DTP in SAP Netweaver BW (BI)

Fig. 4 Semantic Grouping is now available

Update (1/14/2011):

Recently, I had to implement my own version of the obsolete ABAP construct “ON CHANGE OF” event block and found the following to be true: While records may be grouped in a DTP as shown above, you cannot assume that they are sorted by the DSO keys as defined.

You will need to perform an explicit “SORT” on the source/result package as required by your start/end routine logic if you are performing custom comparison operations. Be diligent! =D

Related posts:

  1. How To: Resolve Data Load Issues with Dead Locks
  2. Success! SAP Netweaver BI Administration Cockpit
  3. Estimating Complexity for SAP Netweaver BW BI
  • http://www.levoltz.com Levoltz

    Hi,nDoes this impact the performance of the DTP, by any chance?nThanks,nlevoltz

  • http://www.haungo.com/ Hau Ngo

    Hello Levoltz,nnFrom my experience, the existence of the Semantic Group does not have an impact on the performance of the DTP. nnThere are other parameters in the DTP settings, however, that have an immediate impact on the performance of your ETL process. Each DTP will have settings, such as the number of concurrent packages and the size of each package, that you can adjust and fine-tune for your particular datasource. nnYou can read more about DTP Performance Optimization here: http://help.sap.com/saphelp_nw70/helpdata/en/47/e8c56ecd313c86e10000000a42189c/frameset.htmnnIn additional, I have read an article where the existence of a Start Routine in your transformation will have a negative effect on DTP performance due to the system’s inability to optimize the selection from your source fields. Currently, I do not have a link to that article but I update this comment when possible.nnGood luck!

  • Thevolts

    Hi Hau,nnWe are facing a problem with FI_GL_10. We are extracting 1 million records from R/3 which becomes 10 million when it is updating the DSO. But only less than 1 million get updated in the cube. Could you please help us in this one. nThanks,nKumarn

  • http://www.haungo.com/ Hau Ngo

    Hi Kumar, nnThis may be an issue with your extractor or customer user-exit program. Does the 10 million records consolidate into 1 million record during the activation process in the DSO? nnAlso, try to perform a manual, full-load for a small set of records via RSA3 or verify that they are unique in the PSA. It sounds like you have a LOOP statement that is duplicating your records.

  • Thevolts

    Thanks for the reply. The issue was because for every entry in R/3 it pulled in around 32 records in BW which later gets consolidated. nThe 32 records are due to debit and credit entries and 16 period(which are as columns in R/3).nThanks,nKumar

  • Sapsap

    Hi, Hau Ngo. Thanks for sharing your expirience! It’s really nice blog.nSemantic Group impact on a system performance. Plz, follow this link = nhttp://dev.diditho.net/2011/01/02/sap-bw-dtp-extract-very-large-data-from-psa-to-dso/

  • Hau P Ngo

    Hi Didit,nnGreat write-up on the PSA temp space on your blog. Hopefully, the comments that I left will help you in your set of data extraction and loads!nnHau

  • http://www.haungo.com Hau Ngo

    Hi Didit,

    Great write-up on the PSA temp space on your blog. Hopefully, the comments that I left will help you in your set of data extraction and loads!

    Hau

  • http://www.haungo.com/ Hau Ngo

    Hi Didit,nnGreat write-up on the PSA temp space on your blog. Hopefully, the comments that I left will help you in your set of data extraction and loads!nnHau

  • http://twitter.com/singletrackyyc singletrack

    Thanks for this, it helped me.  Nice site!