Introduction:

Shopify is a fantastic platform, but it doesn’t give you direct access to modify the sitemap.xml file. This can be a problem if you need a customized sitemap with specific URLs, or if you want more control over what search engines crawl. In this article, we’ll explore how to use the Shopify Admin API to generate a custom sitemap, upload it to your store, and make it accessible to search engines. We won’t be modifying Shopify’s default sitemap; instead, we'll create a custom one and make it accessible through /sitemap.xml.

Understanding the Challenge

Shopify auto-generates a sitemap.xml file for your store, and direct modification of this file is not supported. If you require more control over the contents of this file, then this approach will not be enough. While this approach offers a solution using a url redirect, there is an alternative approach that will work for you if you do not want a redirect, see the notes section below for details

The Solution: Using the Shopify API to Generate and Serve a Custom Sitemap

Our solution involves using the Shopify Admin API to gather your website’s resources, generate a custom sitemap.xml file, upload it to your store, and configure a URL redirect. This ensures search engines can access your customized sitemap. Here's a breakdown of the process:

Steps to Generate a Custom Sitemap:

  1. Gather Site Resources using the Shopify API:

    • Use the Shopify Admin API to fetch resources like products, articles, pages, and collections.

    • Create an XML string containing the URLs of these resources in the format required by sitemap.xml

  2. Create the sitemap.xml File:

    • Save the constructed XML string into a file named sitemap.xml.

  3. Upload to Shopify:

    • Utilize the Shopify Admin API to upload the sitemap.xml file to your store's /content/files directory.

  4. Create a URL Redirect (Optional):

    • Set up a URL redirect in Shopify, redirecting /sitemap.xml to the uploaded XML file in /content/files.

  5. Submit to Search Engines (Optional):

    • Submit the URL of the uploaded XML file (located in /content/files) to search engines such as Google Search Console.

Detailed Instructions with Code (Node.js):

  1. Install the Shopify API library:

    npm install @shopify/shopify-api axios

    content_copydownload

    Use code with caution.Bash

  2. Create a Node.js script (e.g., generate-sitemap.js):

    content_copydownload

    Use code with caution.JavaScript

const { ApiVersion, Shopify } = require('@shopify/shopify-api');
const fs = require('fs').promises; // Use promises version of fs
const axios = require('axios');

const shopifyConfig = {
    apiKey: 'YOUR_API_KEY',
    apiSecretKey: 'YOUR_API_SECRET_KEY',
    shop: 'YOUR_SHOP_DOMAIN', // eg: your-shop-name.myshopify.com
    apiVersion: ApiVersion.July23, // Change to your desired API version
};

const shopify = new Shopify(shopifyConfig);

const generateSitemap = async () => {
     try {
        // 1. Fetch all products
        const products = await fetchAllResources('products');
         console.log(`Fetched ${products.length} products`)
       
         // 2. Fetch all articles
        const articles = await fetchAllResources('articles');
         console.log(`Fetched ${articles.length} articles`)
         
         // 3. Fetch all pages
         const pages = await fetchAllResources('pages');
         console.log(`Fetched ${pages.length} pages`)

        // 4. Fetch all collections
        const collections = await fetchAllResources('smart_collections');
         console.log(`Fetched ${collections.length} collections`)

         // 5. Build the XML string
        const xmlString = await buildSitemapXML(products, articles, pages, collections);

        // 6. Generate local sitemap.xml
        const localFilePath = 'sitemap.xml';
        await fs.writeFile(localFilePath, xmlString);
        console.log('sitemap.xml file generated successfully');

         //7.  Upload the sitemap to Shopify
        const remoteFileURL = await uploadSitemapToShopify(localFilePath);
        console.log(`Sitemap uploaded to Shopify: ${remoteFileURL}`);

          // 8. Create URL Redirect
        await createURLRedirect(`/sitemap.xml`, remoteFileURL)
        console.log(`URL redirect created from /sitemap.xml to ${remoteFileURL}`)

         //9. Output results for easy checking
        console.log('Sitemap generation process complete.');
         console.log(`You can access the sitemap using: /sitemap.xml`)

     } catch (error) {
         console.error('Error during sitemap generation:', error);
     }
}

 // Helper function to fetch all resources using pagination
const fetchAllResources = async (resourceType) => {
    let allResources = [];
    let pageInfo = null;
    let hasNextPage = true;

    while (hasNextPage) {
        const params = { limit: 250 };
        if (pageInfo && pageInfo.next) {
            params.page_info = pageInfo.next;
        }

        const session = shopify.session.customAppSession(shopifyConfig.shop);
        const api = new shopify.clients.Rest({ session });

        const response = await api.get({
            path: resourceType,
            query: params,
        });

         if(response.body && response.body[resourceType]){
            allResources = allResources.concat(response.body[resourceType]);
            if(response.body.hasOwnProperty('pagination') && response.body.pagination.hasOwnProperty('next')){
                pageInfo = response.body.pagination
                hasNextPage = pageInfo.next !== null
            }else{
               hasNextPage = false
            }
        }
        else{
          hasNextPage = false
        }

    }

    return allResources;
}

content_copydownload

Use code with caution.

// Helper function to build XML sitemap string
const buildSitemapXML = async (products, articles, pages, collections) => {
const host = shopifyConfig.shop.replace(".myshopify.com", "");
const urlBase = https://${host}.myshopify.com;

let xmlString = `<?xml version="1.0" encoding="UTF-8"?>
                              <urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">`;

        // Add URLs
       
       // Add home page
        xmlString += `<url><loc>${urlBase}</loc><priority>1.0</priority><changefreq>daily</changefreq></url>`;

        // Add products
        products.forEach((product) => {
          xmlString += `<url><loc>${urlBase}/products/${product.handle}</loc><priority>0.8</priority><changefreq>weekly</changefreq></url>`;
         });
       
         // Add articles
         articles.forEach((article) => {
            xmlString += `<url><loc>${urlBase}/blogs/${article.blog_id}/articles/${article.handle}</loc><priority>0.6</priority><changefreq>weekly</changefreq></url>`;
        });

        // Add pages
        pages.forEach((page) => {
            xmlString += `<url><loc>${urlBase}/pages/${page.handle}</loc><priority>0.7</priority><changefreq>weekly</changefreq></url>`;
         });
     
       // Add collections
         collections.forEach((collection) => {
             xmlString += `<url><loc>${urlBase}/collections/${collection.handle}</loc><priority>0.7</priority><changefreq>weekly</changefreq></url>`;
       });

       xmlString += `</urlset>`;
        return xmlString;
    };

// Helper function to upload the generated sitemap to Shopify files
const uploadSitemapToShopify = async (localFilePath) => {
      const session = shopify.session.customAppSession(shopifyConfig.shop);
      const api = new shopify.clients.Rest({ session });
      try {
          const fileContent = await fs.readFile(localFilePath);

          const uploadData = {
            "file": {
              "filename": "sitemap.xml",
              "attachment": fileContent.toString('base64'),
              "content_type": "application/xml"
            }
          };

          const uploadResponse = await api.post({
             path: 'files.json',
              data: uploadData
          });
            if(uploadResponse && uploadResponse.body && uploadResponse.body.hasOwnProperty("file") && uploadResponse.body.file.hasOwnProperty("url")) {
              return uploadResponse.body.file.url
            }else{
              console.log(uploadResponse);
              throw new Error('Error uploading sitemap to Shopify files')
            }
       } catch (error) {
            console.error('Error uploading sitemap to Shopify files:', error);
            throw error;
        }
};

// Helper function to create a URL redirect from /sitemap.xml to uploaded file url
const createURLRedirect = async (path, target) => {
     const session = shopify.session.customAppSession(shopifyConfig.shop);
     const api = new shopify.clients.Rest({ session });
      try{
            const redirectData = {
               "redirect":{
                    "path":path,
                    "target":target
                }
           }
            const response = await api.post({
                path:'redirects.json',
                data:redirectData
            })
            if(response.body && response.body.hasOwnProperty("redirect") && response.body.redirect.hasOwnProperty("id")){
              return true;
            }else{
              throw new Error(`Failed creating url redirect: ${path} => ${target}`)
            }
        }catch(error){
            console.error('Error creating URL redirect:', error);
            throw error;
        }
}

generateSitemap();
```

content_copydownload

Use code with caution.

  1. Configure your Shopify API credentials:

    • Create a custom app in your Shopify admin and obtain your YOUR_API_KEY and YOUR_API_SECRET_KEY.

    • Replace YOUR_SHOP_DOMAIN with your Shopify store’s domain (e.g., your-shop-name.myshopify.com).

    • Update the above code with your actual credentials.

  2. Run the script:

    node generate-sitemap.js

    content_copydownload

    Use code with caution.Bash

Key Code Explanation:

  • shopifyConfig: Stores your API credentials and shop information.

  • fetchAllResources(resourceType): A utility function for fetching resources from shopify with pagination

  • buildSitemapXML(products, articles, pages, collections): Generates the XML sitemap string.

  • uploadSitemapToShopify(localFilePath): Uploads the local file to Shopify files

  • createURLRedirect(path, target): Creates a url redirect

  • generateSitemap(): Main function that runs the whole process

Important Notes:

  • API Rate Limits: The Shopify API has rate limits, be mindful of the speed with which your script makes requests.

  • Error Handling: Ensure your script includes robust error handling.

  • Optimization: Customize the script as per your need, e.g., include support for other resources, modify XML formatting.

  • Security: Securely store and manage your API credentials.

  • Automated Scheduling: Set up a scheduler to run the script regularly for keeping the sitemap updated.

  • Shopify theme modification: If you want a sitemap.xml page without a redirect you can achieve that by modifying the theme files. You can create a liquid template and name it sitemap.xml.liquid and inject the result from the buildSitemapXML() function into the file, and it will be accesible via your-shop-name.myshopify.com/sitemap.xml.

  • sitemap.xml: please note that the generated sitemap does not include product variants, and you need to modify the code to include it if it's required.

Feasibility Analysis:

  • API Viability: The Shopify Admin API allows us to fetch products, articles, pages, collections, upload files and create URL redirects, all officially supported operations.

  • Technical Feasibility: Using Node.js and the Shopify API library makes the implementation straightforward.

  • Effective Results: While we are not directly modifying Shopify's sitemap.xml we can still make it available for search engines by providing the customized sitemap at the /sitemap.xml url

Conclusion:

By using the Shopify API, you can generate a custom sitemap for your Shopify store. Although you can’t directly modify Shopify’s generated sitemap, this approach allows you to host your custom map and either have it redirected to /sitemap.xml, or access it directly by adding the content to the sitemap.xml.liquid template.
This method empowers you to fully control your sitemap and ensure search engines crawl all your important URLs. Remember to tailor the script to suit your needs and to keep it updated.