How to bulk import web pages

This page includes a sample powershell script that will navigate through an Excel file and add each row of information into Interact's Enterprise Search function.

❗️

This is provided as an example only

You will need

  1. An Enterprise Search record in your interact intranet. Instructions for this can be found How to set up Enterprise Search
  2. Some basic understanding of PowerShell scripts and PowerShell software installed on your machine
  3. The web pages to be imported need to be available to the local PC and also available on your company network for users when clicking through from the search results
  4. Create a CSV File with the following values in the first row, then add your list of URLs and relevant data to each row:

How it works

The script below works in the following way

  • Sets up the template for the JSON request to interact

  • Opens the csv file.

  • For each row (after the row 1 header values) the script compiles the JSON data to send to Interact by setting the values based on columns in the csv file.

  • The script then sends the data to Interact to add the web page to the search index.

How to run it

  • The script takes 2 parameters
    1. CSVFile - The path to the CSV file
    2. interactURI - The url to your Interact installation

To run the application cd into the folder that contains the script file and execute the following command

./NetworkFolders.ps1 -CSVFile "<CSV File Location>" -interactURI "<URL to the search app on your intranet>

for example

./NetworkFolders.ps1 -CSVFile "C:\Files\importwebpages.csv" -interactURI "http://demo.interact.com/api/searchapp/1/document”

Things to consider

If you're using this script you need to bear in mind the following things

  • This is not officially supported by Interact, it is intended as a helper and example to what is possible with search apps
  • There is no error detection or data validation
  • When adding files this way there's no checking for duplicates or easy way of deleting documents
  • This will only work as a bulk import, adding new files to the file system will not also add them to Interact
  • You'll need to edit the script to include your own API Key which is added when setting up the search app within Interact
  • If the script is working you should get a success message on screen for each file that is imported

The script

Here it is...

Param(
    [string]$CSVFile,
    [string]$interactURI
)

# Clear Powershell screen
cls

# Build JSON template that will be used for sending to Interact
$templateJSON = "{
  ""Url"": ""{webpageurl}"",
  ""Id"": ""{id}"",
  ""Title"": ""{webtitle}"",
  ""IsPublic"": ""{webispub1ic}"",
  ""Body"": ""{webbody}"",
  ""summary"": ""{websummary}"",
  ""Author"": ""{webauthor}"",
  ""Keywords"": ""{webkeywords}"",
}"

# Point to csv file
$webpageimports=Import-Csv $CSVFile

for ($i=0; $i -lt $webpageimports.Count; $i++) {

    if($webpageimports[$i].URL){

        # Format the Body into one string if is it being treated as multiple rows
        $BodyFormatted = $webpageimports[$i].Body -join " "

        # Update JSON template with information from the CSV file (for each row)
        $fileJSON = $templateJSON 
        $fileJSON = $fileJSON.Replace("{webpageurl}", $webpageimports[$i].URL) 
        $fileJSON = $fileJSON.Replace("{id}", $i) 
        $fileJSON = $fileJSON.Replace("{webtitle}", $webpageimports[$i].Title)
        $fileJSON = $fileJSON.Replace("{webauthor}", $webpageimports[$i].Author)
        $fileJSON = $fileJSON.Replace("{websummary}", $webpageimports[$i].Summary)
        $fileJSON = $fileJSON.Replace("{webkeywords}", $webpageimports[$i].Keywords)
        $fileJSON = $fileJSON.Replace("{webbody}", $BodyFormatted)
        $fileJSON = $fileJSON.Replace("{webispub1ic}", $webpageimports[$i].IsPublic)
      
        #$fileJSON
        
        #Send web page to Interact
        Invoke-RestMethod -Uri $interactURI -Method Put -ContentType "application/json" -Headers @{'X-ApiKey'='<API value>'; } -Body $fileJSON
        
    }
    
}