# Create folder structure  on HDFS Storage

**URL:** <https://forum.ansible.com/t/create-folder-structure-on-hdfs-storage/32930>\
**Category:** Ansible Project\
**Created:** [May 14, 2020, 6:36am UTC](https://forum.ansible.com/t/create-folder-structure-on-hdfs-storage/32930 "2020-05-14T06:36:30Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![dudu.c](https://avatars.discourse-cdn.com/v4/letter/d/278dde/32.png) [@dudu.c](https://forum.ansible.com/u/dudu.c)\
**Post date:** [May 14, 2020, 6:36am UTC](https://forum.ansible.com/t/create-folder-structure-on-hdfs-storage/32930/1 "2020-05-14T06:36:30Z")

</div>

Hi,

I Have a challenge at work that I’m straggling to resolve for few days - Maybe ansible is not the tool for that ???

Requirements: Creating a folder structure on HDFS storage

Detailed Requirements:

1. There are 3 based folders that needs to be created – Let’s assume folder per company

2. On each company folder needs to create X number of folders – Let’s assume that the folder describes departments under each company. For example IT , Rnd , etc.

a. We can assume that all departments can be created for all company folder

1. Under each department, we need to create additional folders that define the assets. For example, computers, network

a. Each department need to be created with relevant assets

1. Last, each asset folder type needs to include additional folders that define the vendor type. For example, in computers, we need to have HP and Dell folders.

a. Each asset type in all departments can be defined with the same vendors

Currently, I have created the company folders (I have also checking if folders exist or not)

From that point, I know how to create the next folders using loop a– only static

**Configuration file :**

## HDFS Base folder

hdfs\_user: hdfs  
hdfs\_company\_folders:

- {path: company1 , user\_owner: admin , group\_owner: company1 , chmod: 640}
- {path: company2 , user\_owner: admin , group\_owner: company2 , chmod: 640}
- {path: company3 , user\_owner: admin , group\_owner: company3 , chmod: 755}

## HDFS departmnet folder

departments:

- {department: IT , department \_folder\_user\_owner: admin , department \_folder\_group\_owner: IT , department \_folder\_chmod: 640}
- { department t: rnd , department \_folder\_user\_owner: admin , department \_folder\_group\_owner: rnd , department \_folder\_chmod: 640}

## HDFS assets & Vendor folder

assets:

- name: computer  
vendor:
- HP
- Dell
- name: network  
vendor:
- cisco

**Create Compony folders – playbook**

1. 
  1. Check what are the folders exists in HDFS

2. 
  1. Update chown and chmod

```auto
3. - hosts: gw

  become: true

  become_user: "{{ hdfs_user }}"

  vars_files:

    - ./hdfs-environment-configuration.yml

  tasks:

   - name: Get folder list from HDFS

     shell: hadoop fs -ls / | awk '{print $8}'

     register: folder_list

   - name: create company folders on HDFS

     shell: hadoop fs -mkdir {{ item.path }}

     register: result

     with_items: "{{ hdfs_company_folders }}"

     when: item.path not in folder_list.stdout_lines

   - name: change base folder user and group owner

     shell: hadoop fs -chown -R {{ item.user_owner }}:{{ item.group_owner }} {{ item.path }}

     with_items: "{{ hdfs_company_folders }}"

     when: item.path not in folder_list.stdout_lines

   - name: change base folder mode

     shell: hadoop fs -chmod {{ item.chmod }} {{ item.path }}

     with_items: "{{ hdfs_company_folders }}"

     when: item.path not in folder_list.stdout_lines

   - name: report which folders were created

     debug:

       msg: "{{ item.path }} folder was created "

     when: item.path not in folder_list.stdout_lines

     with_items: "{{ hdfs_company_folders }}"

```

**Create department folders – playbook** ; i need to understand how I can create everything with variables and loops instead of static

1. 
  1. Get folder list per compony fodder

2. 
  1. Create the relevant department folders and compony folder if not exists

3. 
  1. Report which folder was created

**Create assets and vendor folders – playbook**

1. 
  1. In the configuration file how, I can define which department have which asset type

2. 
  1. Need to create the relevant assets folder under the relevant department

3. 
  1. Need to create a vendor folder under relevant asset folder