Laravel Export Import

A simple and extensible Laravel package for handling large data exports and imports using a queue-based architecture.

This package is designed to export and import large datasets efficiently while keeping memory usage low by leveraging Laravel's LazyCollection and queued jobs.

Features

• Queue based export and import jobs
• Memory efficient processing using LazyCollection
• CSV export support
• Extensible architecture using abstract classes
• Automatic temporary file handling
• Configurable filesystem disk support
• Designed for large datasets

Requirements

• PHP >= 8.3
• Laravel >= 10

Installation

Install the package via Composer.

composer require iqbalatma/laravel-export-import

Publish the configuration file:

php artisan vendor:publish --tag=export-import-config

Publish migrations:

php artisan vendor:publish --tag=migrations

Run migrations:

php artisan migrate

Configuration

After publishing, the configuration file will be available at:

config/export_import.php

Example configuration:

<?php

use Iqbalatma\LaravelExportImport\Models\Export;
use Iqbalatma\LaravelExportImport\Models\Import;
use Iqbalatma\LaravelExportImport\Models\User;

return [

    /*
    |--------------------------------------------------------------------------
    | Model Configuration
    |--------------------------------------------------------------------------
    |
    | Here you may define the models that will be used by the package.
    | This allows you to override the default models with your own
    | implementations if necessary.
    |
    */

    "models" => [
        "user" => User::class,
        "import" => Import::class,
        "export" => Export::class,
    ],

    /*
    |--------------------------------------------------------------------------
    | File Paths
    |--------------------------------------------------------------------------
    |
    | Define the directories used for storing export and import files.
    | Temporary files will be generated during processing before being
    | moved to the final disk storage.
    |
    */

    "path" => [
        "export_path" => "exports",
        "import_path" => "imports",
        "temporary" => "tmp",
    ],

    /*
    |--------------------------------------------------------------------------
    | Import Disk
    |--------------------------------------------------------------------------
    |
    | The disk that will be used to store uploaded import files.
    | This should correspond to one of the disks defined in the
    | "filesystems" configuration file.
    |
    */

    "import_disk" => env("EXPORT_IMPORT_IMPORT_DISK", "s3"),

    /*
    |--------------------------------------------------------------------------
    | Export Disk
    |--------------------------------------------------------------------------
    |
    | The disk where generated export files will be stored.
    | You may configure this to use local, s3, or any supported
    | filesystem disk.
    |
    */

    "export_disk" => env("EXPORT_IMPORT_EXPORT_DISK", "s3"),

    /*
    |--------------------------------------------------------------------------
    | Export Availability (Hours)
    |--------------------------------------------------------------------------
    |
    | Determines how long exported files remain available for download
    | before they expire. The value is defined in hours.
    |
    */

    "export_available_until" => 72,
    
    
    /*
    |--------------------------------------------------------------------------
    | Job Default Timeout (Seconds)
    |--------------------------------------------------------------------------
    |
    | Determines how long job timeout. This value is defined in seconds
    |
    */

    "job_timeout" => 1200,

];

Models

Defines the models used internally by the package.

You may replace these models with your own implementations.

Paths

Defines directories used during import and export processes.

Temporary files are automatically cleaned up after processing.

Export/Import Disk

"export_disk" => "s3"
"import_disk" => "s3"

Specifies which filesystem disk will store import files.

This disk must exist in:

config/filesystems.php

Example disks:

• local
• public
• s3

Export Availability

"export_available_until" => 72

Defines how long exported files remain available for download.

Value is defined in hours.

Examples:

How to export

To start export data you can use trait InteractWithExport

use \Iqbalatma\LaravelExportImport\Traits\InteractWithExport

Example :

<?php

namespace App\Services;

use App\Jobs\Exports\ExportUserJob;
use \Iqbalatma\LaravelExportImport\Traits\InteractWithExport
use Illuminate\Contracts\Container\BindingResolutionException;

class UserService
{
    use InteractWithExport;

    /**
     * @param array $requestedData
     * @return bool
     * @throws BindingResolutionException|AdmissionException|\Throwable
     */
    public static function handle(array $requestedData): bool
    {
        $service = new static();

        $service->createExportEntity(
            exportType: "USER",
            exportName: "Export User"
            permissionName: "can.access.user",
            callback: function () {
                ExportUserJob::dispatch($service->export);
            }
        );
        return true;
    }

}


namespace App\Jobs;

use App\Models\User;
use Illuminate\Support\LazyCollection;
use Iqbalatma\LaravelExportImport\Abstracts\BaseExportJob;

class ExportUserJob extends BaseExportJob
{
    protected LazyCollection $users;

    protected array $header = [
        'name',
        'email'
    ];

    protected function executeQuery(): self
    {
        $this->users = User::query()->lazy(200);

        return $this;
    }

    protected function writeFile(): self
    {
        foreach ($this->users as $user) {
            fputcsv($this->file, [
                $user->name,
                $user->email
            ]);
        }

        return $this;
    }
}

Export Workflow

The export process follows this workflow:

checkIsDirectoryExists
        ↓
setFile
        ↓
executeQuery
        ↓
writeFile
        ↓
exportComplete
        ↓
afterExport

This workflow ensures that exports are handled efficiently and safely.

Export Lifecycle Hooks

You can override lifecycle hooks to customize the export process.

After Export

protected function afterExport(): void
{
    // Example: send notification
}

How to Import

To start export data you can use trait InteractWithImport

use \Iqbalatma\LaravelExportImport\Traits\InteractWithImport

Example :

<?php

namespace App\Services;

use App\Jobs\Exports\ImportUserJob;
use \Iqbalatma\LaravelExportImport\Traits\InteractWithImport
use Illuminate\Contracts\Container\BindingResolutionException;

class UserService
{
    use InteractWithImport;

    /**
     * @param array $requestedData
     * @return bool
     * @throws BindingResolutionException|AdmissionException|\Throwable
     */
    public static function handle(array $requestedData): bool
    {
        $service = new static();

        $file = request()?->file("file");
        if (!$file) {
            throw ValidationException::withMessages(["file" => "Required file not found"]);
        }
        $service->createImportEntity(
            file: $file,
            importType: "USER",
            importName: "Import User"
            permissionName: "can.access.user",
            callback: function () {
                ImportUserJob::dispatch($service->import);
            }
        );
        return true;
    }

}


namespace App\Jobs;

use App\Models\User;
use Illuminate\Support\LazyCollection;
use Iqbalatma\LaravelExportImport\Abstracts\BaseImportJob;

class ImportUserJob extends BaseImportJob
{
    protected function readFile(): self
    {
        $this->getLazyCollection()
            ->chunk(200)
            ->each(function (LazyCollection $collection) {
                foreach ($collection as $row) {
                    try {
                        DB::beginTransaction();
                        #get data from db
                        /** @var User $userFromDB */
                        $userFromDB = User::query()->create($row);

                        $this->successRow++;
                        DB::commit();
                    } catch (Exception $e) {
                        DB::rollBack();
                        $this->generateFileError()
                            ->writeErrorRow($row, $e->getMessage());
                        $this->failedRow++;
                    } finally {
                        $this->totalRow++;
                    }
                }
            });

        return $this;
    }
}

Contributing

Contributions are welcome.

1. Fork the repository
2. Create a new branch
3. Submit a pull request

License

This package is open-sourced software licensed under the MIT license.