--- title: "How to Integrate External Data with the Ansible Lookup" description: "Learn how to fetch data from external sources, like your database, by integrating them with the Ansible lookup in Ansible playbooks in this step-by-step tutorial!" canonical: "https://adamtheautomator.com/ansible-lookup/" --- # How to Integrate External Data with the Ansible Lookup > Learn how to fetch data from external sources, like your database, by integrating them with the Ansible lookup in Ansible playbooks in this step-by-step tutorial! Source: https://adamtheautomator.com/ansible-lookup/ --- ATA Learning Tap to hide [ ATA Learning ](/) * [Home](/) * [Tutorials](/tutorials/) * [Instructors](/author/) * [Advertising](/advertising/) * [Recommended Resources](/resources/) * [About Adam](/about-adam/) Search for: * [](https://twitter.com/adbertram) * [](https://github.com/Adam-the-Automator) * [](https://www.linkedin.com/company/adam-the-automator-llc) * [](/feed/) ![How to Integrate External Data with the Ansible Lookup](https://adamtheautomator.com/wp-content/uploads/2022/02/How-to-Integrate-External-Data-with-the-Ansible-Lookup.jpg) # How to Integrate External Data with the Ansible Lookup [![](https://secure.gravatar.com/avatar/2beb65fca997135120ed98dc6a2e57dcdf1a7d7d2f5ff687b5d91dc7ccd7a6b5?s=192&d=mm&r=g)Sagar](https://adamtheautomator.com/author/shanky-mendiratta/)23 February 20226 min. read Categories: [DevOps](/category/devops/) Tags:[Ansible](/tag/ansible/) Table of Contents * [Prerequisites](#prerequisites) * [Searching for a File using Ansible lookup](#searching-for-a-file-using-ansible-lookup) * [Fetching Environment Variables](#fetching-environment-variables) * [Reading Website Contents](#reading-website-contents) * [Returning Key/Value Pairs with Ansible Dictionary and lookup](#returning-keyvalue-pairs-with-ansible-dictionary-and-lookup) * [Retrieving DNS records with Ansible dig](#retrieving-dns-records-with-ansible-dig) * [Conclusion](#conclusion) While running Ansible playbooks, have you ever wondered how to retrieve data from outside sources, such as files, databases, key/value stores, APIs, and other services? If yes, then exploring [Ansible lookup](https://docs.ansible.com/ansible/latest/user_guide/playbooks_lookups.html) will be worth your while. In this tutorial, you’re going to learn everything about Ansible lookups and how to work with them to fetch external data. Read on and start integrating data! ## Prerequisites This tutorial comprises step-by-step instructions. If you’d like to follow along, be sure you have the following in place: * An [Ansible controller host](https://docs.ansible.com/ansible/latest/installation_guide/index.html) – This tutorial uses [Ansible v2.11.7](https://docs.ansible.com/ansible/latest/installation_guide/intro_installation.html) on an Ubuntu 20.04.3 LTS machine. Related:[How to Setup Ansible (Ubuntu, RHEL, CentOS, macOS)](https://adamtheautomator.com/install-ansible/) * A remote Linux computer to test the tomcat installation – This tutorial uses Ubuntu 20.04.3 LTS as the remote node. * An [inventory file](https://docs.ansible.com/ansible/latest/user_guide/intro_inventory.html) and one or more hosts configured to run Ansible commands and playbooks. The remote Linux computer is called _myserver,_ and this tutorial uses an inventory group called _web_. * NGINX installed on Ansible controller host. * Python v3.6 or later installed both on your Ansible controller host and the remote node machine – This tutorial uses Python v3.9 on an Ubuntu machine. Related:[How Do You Install Python 3.6?](https://adamtheautomator.com/install-python-36/) ## Searching for a File using Ansible `lookup` You typically don’t look up a file and expect that all goes well. What if Ansible can’t find the file, or it doesn’t exist? Use Ansible `lookup` in [Ansible debug](https://docs.ansible.com/ansible/latest/collections/ansible/builtin/debug_module.html) module to manage how errors should treat Ansible tasks. The Ansible debug module prints the statements when you execute an Ansible playbook. In a nutshell, the module allows you to debug variables or expressions without necessarily stopping the playbook. 1\. SSH into your Ansible controller host with a user you typically use to manage Ansible. 2\. Next, create a directory called _ansible\_lookup\_playbook\_demo_ in your home directory. This directory will contain the [playbook](https://www.redhat.com/en/topics/automation/what-is-an-ansible-playbook#:~:text=An%20Ansible%C2%AE%20playbook%20is,make%20up%20an%20Ansible%20inventory.) you’ll invoke later in the section. ```yaml mkdir ~/ansible_lookup_playbook_demo cd ~/ansible_lookup_playbook_demo ``` 3\. Create a file called _main.yml_ in the _~/_ansible\_become\_playbook\_demo directory and copy/paste the following YAML playbook contents. ```yaml --- - name: Ansible Lookup functionality with Ansible debug module demo hosts: web # Executing the tasks with remote_user as ubuntu remote_user: ubuntu tasks: - name: Task ignores the error if shanky.txt does not exist ansible.builtin.debug: msg: "{{ lookup('file', '/shanky.txt', errors='ignore') }}" - name: Task gives warning but continues even if shanky.txt does not exist. ansible.builtin.debug: msg: "{{ lookup('file', '/shanky.txt', errors='warn') }}" - name: Task fails if file doesnt exists ansible.builtin.debug: msg: "{{ lookup('file', '/shanky.txt', errors='strict') }}" ``` 4. Finally, run the below command to invoke the playbook (`main.yml`). Ansible then executes the tasks in the playbook. > _Validating the Ansible playbook using the [`--check`](https://docs.ansible.com/ansible/latest/user_guide/playbooks_checkmode.html#using-check-mode) flag with the `ansible-playbook` command is a good practice before actually executing the playbook. The `--check` flag tells Ansible to perform a simulation without running the playbook._ ```bash ansible-playbook main.yml ``` Below, you can see that all the TASK has an OK status, which indicates the task was not required to execute. Each task did not execute as Ansible couldn’t find the _shanky.txt_ file, and no change was required. ![Executing the Ansible Playbook (main.yml)](https://adamtheautomator.com/wp-content/uploads/2022/02/image-306.png) Executing the Ansible Playbook (_main.yml_) ## Fetching E**nvironment Variables** Earlier in the previous section, you learned how to work with files using Ansible lookup. But at times, you need to read the value of environment variables which are further used in querying URLs or posting as the header or the data in the URL. Create a file called _main2.yml_ in the \*~/\*ansible\_become\_playbook\_demo directory and paste in the following YAML playbook contents. The below Ansible playbook executes two tasks: * The first task checks for the `HOME` environment variable on your ubuntu machine and prints the path of that variable. * Similarly, the second task looks for the `USR` environment variable but displays the `no-user-found` default value as the variable doesn’t exist. ```yaml --- - name: Ansible Lookup with Ansible env module demo hosts: web # Executing the tasks with remote_user as ubuntu remote_user: ubuntu tasks: - name: Checking for enviornment variable HOME ansible.builtin.debug: msg: "'{{ lookup('env', 'HOME') }}' is the HOME environment variable." - name: Checking for enviornment variable USR ansible.builtin.debug: # Ansible task checks for USR variable, if not found takes the default value no-user-found msg: "'{{ lookup('env', 'USR') | default('no-user-found', True) }}' is the user." ``` Once the playbook execution completes, you’ll see that the first task displays the environment path. But in the second task, Ansible can’t find the `USR` user, so Ansible displays the **no-user-found** message. ![Executing an Ansible Playbook to Fetch Environment Variables](https://adamtheautomator.com/wp-content/uploads/2022/02/image-307.png) Executing an Ansible Playbook to Fetch Environment Variables ## Reading Website Contents You previously learned to retrieve data from environment variables stored on your Ubuntu machine using Ansible lookup with Ansible’s built-in debug module. But did you know Ansible lookup works well with multiple other Ansible modules? Some of these Ansible modules are [ansible.builtin.config](https://docs.ansible.com/ansible/latest/collections/ansible/builtin/config_lookup.html#ansible-collections-ansible-builtin-config-lookup), [ansible.builtin.template](https://docs.ansible.com/ansible/latest/collections/ansible/builtin/template_lookup.html#ansible-collections-ansible-builtin-template-lookup), [ansible.builtin.uri](https://docs.ansible.com/ansible/latest/collections/ansible/builtin/uri_module.html), and so on. > _To find out all the modules that work with Ansible lookup, run the `ansible-doc -l -t lookup` command._ For this demo, you’ll work on one of the most widely used Ansible modules, the Ansible URL module ([ansible.builtin.uri](https://docs.ansible.com/ansible/latest/collections/ansible/builtin/uri_module.html)), and learn how it integrates with Ansible lookup. Ansible URL module returns the content of a requested URL to be used as data in an Ansible playbook. Create a file called _main3.yml_ in the \*~/\*ansible\_become\_playbook\_demo directory and paste in the following YAML playbook contents. The ansible-playbook below contains tasks that send requests to a local HTML page and a website to retrieve data. ```yaml --- - name: Ansible Lookup functionality with Ansible URL module demo # Executing the tasks on the inventory group (web) hosts: web # Executing the tasks with remote_user as ubuntu remote_user: ubuntu tasks: - name: Task sends the request to index.html page on nginx installed locally to retrive the data ansible.builtin.debug: msg: "{{ lookup('url', '')}}" - name: Task sends the request to adamtheautomator website and retrive the data ansible.builtin.debug: msg: "{{ lookup('url', '', username='U', password='p') }}" ``` Now, run the below command to execute the Ansible playbook (`main3.yml`), retrieving data from a local page and a live website. ```bash ansible-playbook main3.yml ``` After successful playbook execution, you’ll see both tasks retrieved and printed data (source code) of both the local page and the _[adamtheautomator.com](http://adamtheautomator.com)_ website. ![Requesting and Retrieving Data from Both Local and Live Website](https://adamtheautomator.com/wp-content/uploads/2022/02/image-308.png) Requesting and Retrieving Data from Both Local and Live Website ## R**eturning Key/Value Pairs with Ansible Dictionary and** lookup At times you need to have some environment variables declared in dictionaries format that is key: value format. Reading those values could be tricky, but the combination of Ansible lookup and Ansible dictionary works the magic. Let’s see it in action. Replace the content of the _main.yaml_ playbook you created in the “Searching for a File using the Ansible lookup” section with the content below. In the below Ansible Playbook, Ansible lookup fetches the dictionary in one go; then Ansible dictionary returns a list with each item Ansible lookup requested. ```yaml # Declaring the variable users that further contains two users (alistek and abertram) vars: users: alistek: # users contain two keys: name and telephone # Values of the key name are: Adamlistek and AdamBertram # Values of the key telephone are 1234567890 and 1122334455. name: Adamlistek telephone: 1234567890 abertram: name: AdamBertram telephone: 1122334455 # Ansible task that uses a lookup to display the user's information from the dictionary tasks: - name: Print phone number of employees debug: msg: "User {{ item.key }} is {{ item.value.name }} ({{ item.value.telephone }})" loop: "{{ lookup('dict', users) }}" ``` Now, run the ansible-playbook command below to execute the `main.yml` playbook. ```bash ansible-playbook main.yml ``` As you see below, the Ansible task prints the dictionary items with their values retrieved from the `users` variable. ![Returning Key/Value Pairs in Dictionary Form](https://adamtheautomator.com/wp-content/uploads/2022/02/image-309.png) Returning Key/Value Pairs in Dictionary Form ## Retrieving DNS records with Ansible dig Previously you learned how the Ansible lookup plugin interlinks with various Ansible modules such as Ansible Dictionary, Ansible debugs, Ansible templates, etc. But there are dozens of other Ansible modules that work well with the Ansible Lookup such as [dig](https://docs.ansible.com/ansible/2.9_ja/plugins/lookup/dig.html) plugin. Let’s quickly check out how to work with Ansible dig. > _To find the list of all the Ansible lookup Plugins that are locally installed on your Ansible controller run the `ansible-doc -t lookup -l` command._ Again, replace the content of the _main.yaml_ playbook you created in the previous section with the content below. In the below Ansible Playbook, Ansible dig retrieves the DNS records for [google.com](http://google.com) that Ansible lookup requested. ```yaml --- - name: Ansible dig functionality with Ansible Lookup hosts: localhost # Executing the tasks on the local machine tasks: - name: Retriving DNS records for google.com debug: msg: "{{ lookup ('dig', 'google.com')}}" ``` After you execute ansible playbook as you can the DNS record of `google.com` is displayed on the output. ![DNS record of google.com using Ansible Dig and Lookup](https://adamtheautomator.com/wp-content/uploads/2022/02/image-310.png) DNS record of google.com using Ansible Dig and Lookup ## Conclusion In this tutorial, you’ve taken advantage of the Ansible `lookup` parameter to read external data from various external sources, such as websites environment variables with a single command. Now that you have sound knowledge of the Ansible lookup, which data do you plan to retrieve next on your machine with the Ansible playbook? Share this article [Share on X](https://twitter.com/intent/tweet?url=https%3A%2F%2Fadamtheautomator.com%2Fansible-lookup%2F&text=How%20to%20Integrate%20External%20Data%20with%20the%20Ansible%20Lookup)[Share on Facebook](https://www.facebook.com/sharer/sharer.php?u=https%3A%2F%2Fadamtheautomator.com%2Fansible-lookup%2F)[Share on LinkedIn](https://www.linkedin.com/sharing/share-offsite/?url=https%3A%2F%2Fadamtheautomator.com%2Fansible-lookup%2F) ## Related Posts ![](https://adamtheautomator.com/wp-content/uploads/2021/04/A-Step-by-Step-Guide-to-Getting-Started-with-Ansible-on-Windows.jpg) ### [Mastering Ansible on Windows: Your Go-To Expert Guide](/ansible-on-windows/) Ansible on Windows made simple. A complete guide to hassle-free installation and configuration, perfect for users seeking quick and effective mastery. ![](https://adamtheautomator.com/wp-content/uploads/2022/10/How-to-Manage-Python-Libraries-with-Ansible-Pip.jpg) ### [How to Manage Python Libraries with Ansible Pip](/ansible-pip/) Learn how to effectively manage Python libraries with the Ansible Pip module and take control of your Python dependencies! ![](https://adamtheautomator.com/wp-content/uploads/2022/06/Highly-Effective-Automation-with-Ansible-AWX.jpg) ### [Highly Effective Automation with Ansible AWX](/ansible-awx/) Learn how Ansible AWX can take your Ansible playbooks to the next level and automate all the things with this ATA Learning tutorial! ## Categories * [IT Ops](/category/it-ops/) * [Cloud](/category/cloud/) * [DevOps](/category/devops/) * [Home Ops](/category/home-ops/) * [Information Security](/category/infosec/) * [Software Development](/category/software-development/) ## Site * [Home](/) * [Tutorials](/tutorials/) * [Instructors](/author/) * [Advertising](/advertising/) * [Recommended Resources](/resources/) * [About Adam](/about-adam/) Copyright 2026© ATA Learning | [Privacy Policy](/privacy/)