File Deduplication (Docker)

This is a simple project designed to help manage and clean up directories by identifying and removing duplicate files and empty directories. It scans specified root directories, finds duplicate files based on their content, logs file details, and optionally deletes them based on filenames and dates.

This project uses Go libraries like logrus for logging and caarlos0/env for handling environment variables. It first finds all files with the same sizes, then relies on MD5 and SHA-512 hashing to spot duplicate files accurately.

The project is designed to run in a container with configurable environment variables:

services:
  dedup:
    image: meinya/dedup:latest
    volumes:
        - /to_dedup:/dedup
    environment:
        - DO_REMOVE=false
        - MIN_SIZE=100000000
        - EXCLUDE_EXT=.3fr