How to backup and restore Glue data catalog
2020-02-21
How to recover a wrongly deleted glue table? You should have scheduled a periodic backup of Glue data catalog with
aws glue get-tables --database-name mydb > glue-mydb.json
And recreate your table with the command
aws glue create-table --cli-input-json '{...}'
But the json format of aws glue get-tables is quite different from the json format of aws create-table. For the conversion you can use a simple python script like the following one
import json, sys
def dict_convert(dict):
DatabaseName = dict['DatabaseName']
del dict['DatabaseName']
del dict['CreateTime']
del dict['UpdateTime']
del dict['IsRegisteredWithLakeFormation']
result = {
'DatabaseName': DatabaseName,
'TableInput': dict
}
return result
data = json.load(sys.stdin)
for t in data["TableList"]:
print(json.dumps(dict_convert(t), default=str))