Skip to content

Commit

Permalink
Merge pull request #293 from humanmade/fix-upload-directory
Browse files Browse the repository at this point in the history 
Remove upload-attachments WP CLI command and fix upload-directory
  • Loading branch information
@joehoyle
joehoyle authored Apr 5, 2019
2 parents 6665b6e + fef7efb commit 09b2062
Show file tree
Hide file tree
Showing 4 changed files with 30 additions and 193 deletions.
31 changes: 13 additions & 18 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ define( 'S3_UPLOADS_REGION', '' ); // the s3 bucket region (excluding the rest o
```
Please refer to this region list http://docs.aws.amazon.com/general/latest/gr/rande.html#s3_region for the S3_UPLOADS_REGION values.

Use of path prefix after the bucket name is allowed and is optional. For example, if you want to upload all files to 'my-folder' inside a bucket called 'my-bucket', you can use:
Use of path prefix after the bucket name is allowed and is optional. For example, if you want to upload all files to 'my-folder' inside a bucket called 'my-bucket', you can use:

```PHP
define( 'S3_UPLOADS_BUCKET', 'my-bucket/my-folder' );
Expand All @@ -53,8 +53,8 @@ You must then enable the plugin. To do this via WP-CLI use command:
wp plugin activate S3-Uploads
```

The plugin name must match the directory you have cloned S3 Uploads into;
If you're using Composer, use
The plugin name must match the directory you have cloned S3 Uploads into;
If you're using Composer, use
```
wp plugin activate s3-uploads
```
Expand All @@ -75,18 +75,7 @@ wp s3-uploads create-iam-user --admin-key=<key> --admin-secret=<secret>

This will provide you with a new Access Key and Secret Key which you can configure S3-Uploads with. Paste the values in the `wp-config.php`. Once you have migrated your media to S3 with any of the below methods, you'll want to enable S3 Uploads: `wp s3-uploads enable`.

If you want to create your IAM user yourself, or attach the neccessary permissions to an existing user, you can output the policy via `wp s3-uploads generate-iam-policy`

Migrating your Media to S3
==========

S3-Uploads can migrate your existing media library to S3. Once you have S3-Uploads up and running, use the following WP-CLI command:

```
wp s3-uploads migrate-attachments [--delete-local]
```

By default, S3-Uploads will keep your files locally just incase something goes wrong, but you can delete with the `--delete-local` flag.
If you want to create your IAM user yourself, or attach the necessary permissions to an existing user, you can output the policy via `wp s3-uploads generate-iam-policy`


Listing files on S3
Expand All @@ -101,14 +90,20 @@ wp s3-uploads ls [<path>]
Uploading files to S3
==========

Sometimes the `wp s3-uploads migrate-attachments` command may not be enough to migrate your uploads to S3, as that will only move attachment files to S3. If you are using any plugins that store data in uploads, you'll want to upload the whole `uploads` directory.
If you have an existing media library with attachment files, use the below command to copy them all to S3 from local disk.

```
wp s3-uploads upload-directory <from> <to> [--sync] [--dry-run]
wp s3-uploads upload-directory <from> <to> [--verbose]
```

Passing `--sync` will only upload files that are newer in `<from>` or that don't exist on S3 already. Use `--dry-run` to test.

For example, to migrate your whole uploads directory to S3, you'd run:

```
wp s3-uploads upload-directory /path/to/uploads/ uploads
```

There is also an all purpose `cp` command for arbitrary copying to and from S3.

```
Expand Down Expand Up @@ -172,7 +167,7 @@ S3 Object Permissions

The object permission of files uploaded to S3 by this plugin can be controlled by setting the `S3_UPLOADS_OBJECT_ACL`
constant. The default setting if not specified is `public-read` to allow objects to be read by anyone. If you don't
want the uploads to be publicly readable then you can define `S3_UPLOADS_OBJECT_ACL` as one of `private` or `authenticated-read`
want the uploads to be publicly readable then you can define `S3_UPLOADS_OBJECT_ACL` as one of `private` or `authenticated-read`
in you wp-config file:

```PHP
Expand Down
25 changes: 0 additions & 25 deletions inc/class-s3-uploads-changed-files-iterator.php
View file

This file was deleted.

44 changes: 0 additions & 44 deletions inc/class-s3-uploads-uploadsyncbuilder.php
View file

This file was deleted.

123 changes: 17 additions & 106 deletions inc/class-s3-uploads-wp-cli-command.php
View file
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
<?php

use Aws\S3\Transfer;

class S3_Uploads_WP_CLI_Command extends WP_CLI_Command {

/**
Expand Down Expand Up @@ -53,92 +55,6 @@ public function verify_api_keys() {
WP_CLI::success( 'Looks like your configuration is correct.' );
}

/**
* @subcommand migrate-attachments
* @synopsis [--delete-local]
*/
public function migrate_attachments_to_s3( $args, $args_assoc ) {

$attachments = new WP_Query( array(
'post_type' => 'attachment',
'posts_per_page' => -1,
'post_status' => 'all',
));

WP_CLI::line( sprintf( 'Attempting to move %d attachments to S3', $attachments->found_posts ) );

foreach ( $attachments->posts as $attachment ) {

$this->migrate_attachment_to_s3( array( $attachment->ID ), $args_assoc );
}

WP_CLI::success( 'Moved all attachments to S3. If you wish to update references in your database run: ' );
WP_CLI::line( '' );

// Ensure things are active
$instance = S3_Uploads::get_instance();

if ( ! s3_uploads_enabled() ) {
$instance->setup();
}

$old_upload_dir = $instance->get_original_upload_dir();
$upload_dir = wp_upload_dir();

WP_CLI::Line( sprintf( 'wp search-replace "%s" "%s"', $old_upload_dir['baseurl'], $upload_dir['baseurl'] ) );
}

/**
* Migrate a single attachment's files to S3
*
* @subcommand migrate-attachment
* @synopsis <attachment-id> [--delete-local]
*/
public function migrate_attachment_to_s3( $args, $args_assoc ) {

// Ensure things are active
$instance = S3_Uploads::get_instance();
if ( ! s3_uploads_enabled() ) {
$instance->setup();
}

$old_upload_dir = $instance->get_original_upload_dir();
$upload_dir = wp_upload_dir();

$files = array();
$attached_file = get_post_meta( $args[0], '_wp_attached_file', true );

if ( ! empty( $attached_file ) ) {
$files[] = $attached_file;
}

$meta_data = wp_get_attachment_metadata( $args[0] );

if ( ! empty( $meta_data['sizes'] ) ) {
foreach ( $meta_data['sizes'] as $file ) {
$files[] = path_join( dirname( $meta_data['file'] ), $file['file'] );
}
}

foreach ( $files as $file ) {
if ( file_exists( $path = $old_upload_dir['basedir'] . '/' . $file ) ) {

if ( ! copy( $path, $upload_dir['basedir'] . '/' . $file ) ) {
WP_CLI::line( sprintf( 'Failed to moved %s to S3', $file ) );
} else {
if ( ! empty( $args_assoc['delete-local'] ) ) {
unlink( $path );
}
WP_CLI::success( sprintf( 'Moved file %s to S3', $file ) );

}
} else {
WP_CLI::line( sprintf( 'Already moved to %s S3', $file ) );
}
}

}

/**
* Create an AWS IAM user for S3 Uploads to user
*
Expand Down Expand Up @@ -299,7 +215,7 @@ public function cp( $args ) {
* Upload a directory to S3
*
* @subcommand upload-directory
* @synopsis <from> [<to>] [--sync] [--dry-run] [--concurrency=<concurrency>]
* @synopsis <from> [<to>] [--concurrency=<concurrency>] [--verbose]
*/
public function upload_directory( $args, $args_assoc ) {

Expand All @@ -310,26 +226,21 @@ public function upload_directory( $args, $args_assoc ) {
}

$s3 = S3_Uploads::get_instance()->s3();
$bucket = strtok( S3_UPLOADS_BUCKET, '/' );
$prefix = '';

if ( strpos( S3_UPLOADS_BUCKET, '/' ) ) {
$prefix = trailingslashit( str_replace( strtok( S3_UPLOADS_BUCKET, '/' ) . '/', '', S3_UPLOADS_BUCKET ) );
}

$args_assoc = wp_parse_args( $args_assoc, [ 'concurrency' => 5, 'verbose' => false ] );

$transfer_args = [
'concurrency' => $args_assoc['concurrency'],
'debug' => (bool) $args_assoc['verbose'],
'before' => function ( AWS\Command $command ) {
if ( in_array( $command->getName(), [ 'PutObject', 'CreateMultipartUpload' ], true ) ) {
$acl = defined( 'S3_UPLOADS_OBJECT_ACL' ) ? S3_UPLOADS_OBJECT_ACL : 'public-read';
$command['ACL'] = $acl;
}
},
];
try {
$s3->uploadDirectory(
$from,
$bucket,
$prefix . $to,
array(
'debug' => true,
'params' => array( 'ACL' => 'public-read' ),
'builder' => new S3_Uploads_UploadSyncBuilder( ! empty( $args_assoc['dry-run'] ) ),
'force' => empty( $args_assoc['sync'] ),
'concurrency' => ! empty( $args_assoc['concurrency'] ) ? $args_assoc['concurrency'] : 5,
)
);
$manager = new Transfer( $s3, $from, 's3://' . S3_UPLOADS_BUCKET . '/' . $to, $transfer_args );
$manager->transfer();
} catch ( Exception $e ) {
WP_CLI::error( $e->getMessage() );
}
Expand Down

0 comments on commit 09b2062

Please sign in to comment.