Question
Infosys
AU
Last activity: 6 Nov 2018 20:02 EST
Robotics [RDA] - Creating an XML document by scraping a website
Hi all,
I am trying to scrape a website which has a list of address cards. After the scraping is done, I need to send the data over to PEGA where I can parse it and display on UI. I would like to have the data transfer format as XML. So, I have tried 2 options
- Append all those string using some delimiters and later call a C# script component to parse that and build XML
- Use XML Builder [something like XMLDocumentComponent] to do the same.
- This would eliminate custom coding in favor of standards but I could not figure how to build the XML.
- I have tried using a Schema Provider on the XML component so that a proper structure will be defined before I start using it
- Then, using Create.Type method, I tried creating a child node for each address (from ListLoop component)
- Finally, I have tried reading the OuterXML but got nothing
Even though point 1 worked for now, I would really want that custom code to be eliminated if point 2 works. Please suggest...
Accepted Solution
Pegasystems Inc.
US
The simplest way to do this is to put all of your data into a Lookup Table. When you are done you can call the WriteXML method of the underlying DataTable to get a properly formatedd Xml document.
Infosys
AU
Thank you Jeff. That's very convenient.
I completely missed that method in the DataTable class and ended up writing my own Script component with accepts datatable and returns XML. I can now remove the component in favor of this solution.
Capgemini India PVT Ltd
IN
Its not working on my end. Below is code snap.
Pegasystems Inc.
US
You are not passing a value for the writer parameter of the dataTableProxy1.WriteXML() method (according to your your screenshot).
Capgemini India PVT Ltd
IN
In my code, I have added the data into lookup table then executing the proxy object method. If in case, could you share the sample code snap or exact change you are suggesting.
Appreciate your support.
Capgemini India PVT Ltd
IN
Can someone please share sample code snap / project to generate xml string in Pega Robotics. As I need to pass the generated xml to SQL database to store.
Infosys
AU
Hi Manish,
As pointed out by sasnt, you need to send the writer object that that automation block. Else, choose a different overloaded method which accepts a string (filename) and generate XML into that file [which you can read back later]. This is the way I have tried. Attached is the sample.
But for production purposes, you probably need to derive temp location and the best way obviously is to figure out the stream input method.
Infosys
AU
Hi sasnt / Jeff
Could you suggest how to create the Stream (System.IO.MemoryStream) object so that I can pass it to WriteXML method? Creating Objects for .net classes in Automation designer - Is it possible?
**EDIT: After implementing the above in a script block, I now think that not everything should be fitted into an automation designer. A (reusable) custom developed DLL with a static utility method seems to be a good fit for this. (unless there is a single automation block which can achieve the same) -- thoughts please