How to backup and restore Glue data catalog


How to recover a wrongly deleted glue table? You should have scheduled a periodic backup of Glue data catalog with

aws glue get-tables --database-name mydb > glue-mydb.json

And recreate your table with the command

aws glue create-table --cli-input-json '{...}'

But the json format of aws glue get-tables is quite different from the json format of aws create-table. For the conversion you can use a simple python script like the following one

import json, sys

def dict_convert(dict):
    DatabaseName = dict['DatabaseName']
    del dict['DatabaseName']
    del dict['CreateTime']
    del dict['UpdateTime']
    del dict['IsRegisteredWithLakeFormation']
    result = {
        'DatabaseName': DatabaseName,
        'TableInput': dict
    return result

data = json.load(sys.stdin)

for t in data["TableList"]:
    print(json.dumps(dict_convert(t), default=str))

Enter your instance's address

More posts like this

My Networking Survival Kit

2020-03-15 | #Me

In this small tutorial I’ll speak about tunneling, ssh port forwarding, socks, pac files, Sshuttle I’ve been using Linux since 1995 but I have never been interested a lot in networking.

Continue reading 