A Custom Python App to Speed Up Salesforce Marketing Cloud Lookups

If you've spent time working with Salesforce Marketing Cloud (SFMC), you're probably familiar with Contact Builder and the process of mapping Data Extensions (DEs) to perform lookups. While Contact Builder is a powerful tool, it can also be slow and cumbersome, especially when you're trying to do something as simple as checking a single value across multiple DEs.

In this post, I'll share a custom Flask app I built that offers a faster, more flexible alternative to Contact Builder for multi-DE lookups.

The Problem with using Contact Builder for Lookups

Using Contact Builder for lookups can be a frustrating experience, particularly when:

• Performance is slow: Contact Builder isn't exactly known for its speed. Loading the UI, mapping Data Extensions, and then running lookups can take far more time than you'd like.

• Mapping limitations: To use Contact Builder, you have to map each Data Extension you want to work with. This might be fine for a few key DEs, but when you're dealing with temporary or test DEs, this step becomes impractical—especially in a production environment where data may be sensitive or frequently changing.

• Overkill for simple tasks: If you're just trying to look up a single value across multiple Data Extensions, Contact Builder can feel like using a sledgehammer to crack a nut. It's designed for complex relationships and segmentation, but not for quick, one-off lookups.

The Simple Alternative: A Custom Lookup App

To solve the issues with Contact Builder, I created a custom Flask app that allows you to perform fast lookups across multiple Data Extensions with just one input value. Here's how it improves on Contact Builder:

• Faster lookups, fewer steps: Instead of waiting for Contact Builder to load, this app allows you to submit a lookup value and get your results in seconds.

• No mapping required: The app reads from a simple lookups.json file that lists the Data Extensions and the fields you want to search. You don't have to map these DEs in Contact Builder, which makes it perfect for production environments or when working with temporary test data.

• Works with any DE: Whether you're working with live production data or test Data Extensions, the app can retrieve the information you need without the hassle of Contact Builder.

Simple JSON configuration file for defining lookup targets

Clean, simple lookup interface

Fast, tabular results showing data from multiple Data Extensions

How the App Works

The app is built using Flask and connects to SFMC using the SOAP API. Here's a quick look at how it works:

1. Configuration: You define the Data Extensions and the lookup fields in a simple JSON file (lookups.json).

2. User Input: The app provides a simple web form where you enter a single lookup value (like a SubscriberKey or an OrderNumber).

3. Results: The app retrieves the data from each Data Extension listed in lookups.json and displays it in a clean table format.

Example Use Case

Imagine you're trying to find all instances of a subscriber using their SubscriberKey. Instead of having to map every relevant Data Extension in Contact Builder, you can define these DEs in the app's lookups.json file, enter the SubscriberKey in the form, and get instant results—without any additional configuration.

Setting It Up

Setting up the app is straightforward. You can clone the repo from GitHub and follow these steps:

1. Clone the Repository

Start by cloning the repository:

1git clone https://github.com/lucasCostaYVR/sfmc-helper.git
2cd sfmc-helper

2. Set Up a Virtual Environment

It's best to isolate your dependencies with a virtual environment:

1python3 -m venv venv
2source venv/bin/activate  # On macOS/Linux
3venv\Scripts\activate     # On Windows

3. Install Dependencies

Once your virtual environment is active, install the required Python packages:

1pip install -r requirements.txt

4. Create a .env File

You'll need to create a .env file with your Salesforce Marketing Cloud credentials and base URLs. Here's an example format:

1CLIENT_ID=your-client-id
2CLIENT_SECRET=your-client-secret
3AUTH_BASE_URL=https://your-auth-url.auth.marketingcloudapis.com/
4REST_BASE_URL=https://your-rest-url.rest.marketingcloudapis.com/
5SOAP_BASE_URL=https://your-soap-url.soap.marketingcloudapis.com/
6FLASK_ENV=development

5. Edit lookups.json

Customize the lookups.json file to include the Data Extensions you want to search. It should look something like this:

1[
2  {
3      "ExternalKey": "SubscriberKeyLookup",
4      "LookupField": "SubscriberKey"
5  },
6  {
7      "ExternalKey": "OrderLookup",
8      "LookupField": "OrderNumber"
9  }
10]

6. Run the App

Now you can run the Flask app:

1python app.py

The app will be available at http://localhost:5000 where you can start performing lookups immediately.

Key Features

Simple Configuration

The lookups.json file makes it easy to configure which Data Extensions to search:

1[
2  {
3      "ExternalKey": "CustomerData",
4      "LookupField": "CustomerID"
5  },
6  {
7      "ExternalKey": "OrderHistory",
8      "LookupField": "CustomerID"
9  },
10  {
11      "ExternalKey": "SupportTickets",
12      "LookupField": "CustomerID"
13  }
14]

Fast API Integration

The app uses the SFMC SOAP API for reliable, fast data retrieval. Unlike Contact Builder's UI-heavy approach, this direct API connection means:

• No UI loading delays
• Instant data retrieval
• Batch processing of multiple DEs
• Reliable connection handling

Flexible Field Mapping

You can configure different lookup fields for different Data Extensions, making it versatile for various use cases:

• Customer lookups across multiple systems
• Order tracking across different databases
• Subscriber journey mapping
• Data quality auditing

Why This App Saves Time

• Faster lookups: With this app, you don't have to wait for Contact Builder to process your query. Just input the value, and the results come up almost instantly.

• No need to map DEs: One of the biggest time-savers is the ability to work with Data Extensions directly, without having to map them in Contact Builder. This makes it easier to work in production environments, where you may not want to set up mappings for every DE.

• Perfect for production and test environments: Whether you're troubleshooting live production data or experimenting with test Data Extensions, the app gives you flexibility that Contact Builder simply doesn't.

Limitations and Next Steps

This app was built with the goal of simplifying lookups in Data Extensions, making it a handy tool for daily activities, data investigations, and QA. However, there are a few limitations to keep in mind:

• Performance on large datasets: As the number of Data Extensions grows, the lookup process may become slower. This is because each lookup is performed sequentially, which can lead to delays when working with a large number of DEs.

• Single-value lookups only: Currently, the app focuses on performing lookups using a single value. More complex queries and filtering options aren't supported yet.

Future Enhancements

To address the performance issue, future versions will include:

1. Asynchronous Processing: Making lookup processes parallel to speed up multi-DE searches
2. Advanced Filtering: Support for complex queries and multiple search criteria
3. Caching: Local caching of frequently accessed data
4. Export Features: CSV/Excel export of lookup results
5. Audit Logging: Track and log all lookup activities

Use Cases

This tool is particularly valuable for:

Data Quality Auditing

• Finding duplicate records across multiple Data Extensions
• Validating data consistency between systems
• Identifying orphaned records

Customer Support

• Quickly locating customer data across all touchpoints
• Troubleshooting data issues in real-time
• Providing rapid response to customer inquiries

Development and Testing

• Validating data flows in development environments
• Testing data integration without affecting production mappings
• Quick data verification during deployment

Marketing Operations

• Subscriber journey tracking across multiple Data Extensions
• Campaign performance analysis
• Data investigation for optimization opportunities

Conclusion

If you've been frustrated with the slowness or limitations of Contact Builder, this Flask app could be a game-changer. It's fast, flexible, and allows you to perform lookups across multiple Data Extensions in your SFMC environment without the need for complex mappings. Whether you're working with production data or conducting quick tests, this app can streamline your workflow.

While there are still some limitations, like performance with larger DEs, future updates will bring asynchronous lookups to make the app even faster. The tool is available on GitHub for you to try and customize according to your needs.

Give it a try, and feel free to share your feedback—your input will help shape the next round of features!