I made a plugin that basically reads a CSV file and imports data to relevant tables.
However, the action seems to create an error:
Fatal error: Allowed memory size of 134217728 bytes exhausted (tried to allocate 65015808 bytes) in /var/www/proj/wp-includes/functions.php on line
which led me to this code in functions.php:
function wp_ob_end_flush_all() {
$levels = ob_get_level();
for ($i=0; $i<$levels; $i++)
ob_end_flush();
}
I did a Google and came across two popular solutions, both of which didn't seem to work.
Solution 1: disabling zlib - this is already disabled.
Solution 2:remove_action('shutdown', 'wp_ob_end_flush_all', 1);
Solution 2 still errors but with no message, which, isn't exactly ideal.
This is the script that's causing the error:
<?php
ini_set('display_startup_errors', 1);
ini_set('display_errors', 1);
error_reporting(-1);
# load core wp fnc
require_once $_SERVER['DOCUMENT_ROOT']. '/wp-load.php';
# load db functions
require_once $_SERVER['DOCUMENT_ROOT']. '/wp-admin/includes/upgrade.php';
# load admin fnc
require_once $_SERVER['DOCUMENT_ROOT']. '/wp-content/plugins/vendor-module/admin/inc/admin-functions.php';
global $wpdb;
$admin_functions = new vendor_module_admin_functions();
# get csv
$file = $_FILES['csv'];
$name = $file['name'];
$dir = $file['tmp_name'];
# rm spaces, replace with _
$name = str_replace(' ', '_', $name);
$file_location = $_SERVER['DOCUMENT_ROOT']. '/wp-content/plugins/vendor-module/uploads/import/'. $name;
# if successfully moved, carry on, else return
$moved = ($file['error'] == 0 ? true : false);
$error = false;
if (!$moved) {
echo 'Error! CSV file may be incorrectly formatted or there was an issue in reading the file. Please try again.';
} else {
move_uploaded_file($dir, $file_location);
$id = $_POST['val'];
$type = $_POST['type'];
$table = ($type == 1 ? 'vendor_module_type_one' : 'vendor_module_type_two');
$handle = fopen($file_location, 'r');
$parts = array();
$components = array();
$i = 0;
while (($data = fgetcsv($handle, 1000, ',')) !== false)
{
if (is_array($data)) {
if (empty($data[0])) {
echo 'Error! Reference can\'t be empty. Please ensure all rows are using a ref no.';
$error = true;
continue;
}
# get data
$get_for_sql = 'SELECT `id` FROM `'. $wpdb->prefix. $table .'` WHERE `ref` = %s';
$get_for_res = $wpdb->get_results($wpdb->prepare($get_for_sql, array($data[0])));
if (count($get_for_res) <= 0) {
echo 'Error! Reference has no match. Please ensure the CSV is using the correct ref no.';
$error = true;
exit();
}
$get_for_id = $get_for_res[0]->id;
# create data arrays
$parts[$i]['name'] = $data[1];
$parts[$i]['ref'] = $data[2];
$parts[$i]['for'] = $get_for_id;
$components[$i]['part_ref'] = $data[2];
$components[$i]['component_ref'] = $data[3];
$components[$i]['sku'] = $data[4];
$components[$i]['desc'] = utf8_decode($data[5]);
$components[$i]['req'] = $data[6];
$components[$i]['price'] = $data[7];
unset($get_for_id);
unset($get_for_res);
unset($get_for_sql);
$i++;
}
}
fclose($handle);
unlink($file_location);
# get unique parts only
$parts = array_unique($parts, SORT_REGULAR);
# check to see if part already exists, if so delete
$search_field = ($type == 1 ? 'id_field_one' : 'id_field_two');
$check_sql = 'SELECT `id` FROM `'. $wpdb->prefix .'vendor_module_parts` WHERE `'. $search_field .'` = %d';
$delete_parts_sql = 'DELETE FROM `'. $wpdb->prefix .'vendor_module_parts` WHERE `'. $search_field .'` = %d';
$delete_components_sql = 'DELETE FROM `'. $wpdb->prefix .'vendor_module_components` WHERE `part_id` = %d';
$check_res = $wpdb->get_results($wpdb->prepare($check_sql, array($id)));
if ($check_res) {
$wpdb->query($wpdb->prepare($delete_parts_sql, array($id)));
}
$part_ids = $admin_functions->insert_parts($parts, $type);
unset($parts);
unset($delete_parts_sql);
unset($search_field);
unset($check_sql);
unset($check_res);
unset($i);
# should never be empty, but just as a precaution ...
if (!empty($part_ids)) {
foreach ($components as $key => $component)
{
$components[$key]['part_id'] = $part_ids[$component['part_ref']];
}
# rm components from assoc part id
foreach ($part_ids as $id)
{
$wpdb->query($wpdb->prepare($delete_components_sql, array($id)));
}
# insert components
$admin_functions->insert_components($components);
} else {
echo 'Error!';
}
echo (!$error ? 'Success! File Successfully Imported.' : 'There be something wrong with the import. Please try again.');
}
it's triggered through a button press and uses AJAX to handle it etc.
I'm not sure why a memory leak is occurring or why WordPress doesn't offer more useful error messages. I don't call that function.. so I'm guessing it's something WordPress is doing in the background when things are run.
My info:
PHP 7.2.10
Apache 2.4
Linux Mint 19
Also happens on my server:
PHP 7.1.25
Apache 2.4
CentOS 7.6.1810
WordPress running version: 4.9.8
I made a plugin that basically reads a CSV file and imports data to relevant tables.
However, the action seems to create an error:
Fatal error: Allowed memory size of 134217728 bytes exhausted (tried to allocate 65015808 bytes) in /var/www/proj/wp-includes/functions.php on line
which led me to this code in functions.php:
function wp_ob_end_flush_all() {
$levels = ob_get_level();
for ($i=0; $i<$levels; $i++)
ob_end_flush();
}
I did a Google and came across two popular solutions, both of which didn't seem to work.
Solution 1: disabling zlib - this is already disabled.
Solution 2:remove_action('shutdown', 'wp_ob_end_flush_all', 1);
Solution 2 still errors but with no message, which, isn't exactly ideal.
This is the script that's causing the error:
<?php
ini_set('display_startup_errors', 1);
ini_set('display_errors', 1);
error_reporting(-1);
# load core wp fnc
require_once $_SERVER['DOCUMENT_ROOT']. '/wp-load.php';
# load db functions
require_once $_SERVER['DOCUMENT_ROOT']. '/wp-admin/includes/upgrade.php';
# load admin fnc
require_once $_SERVER['DOCUMENT_ROOT']. '/wp-content/plugins/vendor-module/admin/inc/admin-functions.php';
global $wpdb;
$admin_functions = new vendor_module_admin_functions();
# get csv
$file = $_FILES['csv'];
$name = $file['name'];
$dir = $file['tmp_name'];
# rm spaces, replace with _
$name = str_replace(' ', '_', $name);
$file_location = $_SERVER['DOCUMENT_ROOT']. '/wp-content/plugins/vendor-module/uploads/import/'. $name;
# if successfully moved, carry on, else return
$moved = ($file['error'] == 0 ? true : false);
$error = false;
if (!$moved) {
echo 'Error! CSV file may be incorrectly formatted or there was an issue in reading the file. Please try again.';
} else {
move_uploaded_file($dir, $file_location);
$id = $_POST['val'];
$type = $_POST['type'];
$table = ($type == 1 ? 'vendor_module_type_one' : 'vendor_module_type_two');
$handle = fopen($file_location, 'r');
$parts = array();
$components = array();
$i = 0;
while (($data = fgetcsv($handle, 1000, ',')) !== false)
{
if (is_array($data)) {
if (empty($data[0])) {
echo 'Error! Reference can\'t be empty. Please ensure all rows are using a ref no.';
$error = true;
continue;
}
# get data
$get_for_sql = 'SELECT `id` FROM `'. $wpdb->prefix. $table .'` WHERE `ref` = %s';
$get_for_res = $wpdb->get_results($wpdb->prepare($get_for_sql, array($data[0])));
if (count($get_for_res) <= 0) {
echo 'Error! Reference has no match. Please ensure the CSV is using the correct ref no.';
$error = true;
exit();
}
$get_for_id = $get_for_res[0]->id;
# create data arrays
$parts[$i]['name'] = $data[1];
$parts[$i]['ref'] = $data[2];
$parts[$i]['for'] = $get_for_id;
$components[$i]['part_ref'] = $data[2];
$components[$i]['component_ref'] = $data[3];
$components[$i]['sku'] = $data[4];
$components[$i]['desc'] = utf8_decode($data[5]);
$components[$i]['req'] = $data[6];
$components[$i]['price'] = $data[7];
unset($get_for_id);
unset($get_for_res);
unset($get_for_sql);
$i++;
}
}
fclose($handle);
unlink($file_location);
# get unique parts only
$parts = array_unique($parts, SORT_REGULAR);
# check to see if part already exists, if so delete
$search_field = ($type == 1 ? 'id_field_one' : 'id_field_two');
$check_sql = 'SELECT `id` FROM `'. $wpdb->prefix .'vendor_module_parts` WHERE `'. $search_field .'` = %d';
$delete_parts_sql = 'DELETE FROM `'. $wpdb->prefix .'vendor_module_parts` WHERE `'. $search_field .'` = %d';
$delete_components_sql = 'DELETE FROM `'. $wpdb->prefix .'vendor_module_components` WHERE `part_id` = %d';
$check_res = $wpdb->get_results($wpdb->prepare($check_sql, array($id)));
if ($check_res) {
$wpdb->query($wpdb->prepare($delete_parts_sql, array($id)));
}
$part_ids = $admin_functions->insert_parts($parts, $type);
unset($parts);
unset($delete_parts_sql);
unset($search_field);
unset($check_sql);
unset($check_res);
unset($i);
# should never be empty, but just as a precaution ...
if (!empty($part_ids)) {
foreach ($components as $key => $component)
{
$components[$key]['part_id'] = $part_ids[$component['part_ref']];
}
# rm components from assoc part id
foreach ($part_ids as $id)
{
$wpdb->query($wpdb->prepare($delete_components_sql, array($id)));
}
# insert components
$admin_functions->insert_components($components);
} else {
echo 'Error!';
}
echo (!$error ? 'Success! File Successfully Imported.' : 'There be something wrong with the import. Please try again.');
}
it's triggered through a button press and uses AJAX to handle it etc.
I'm not sure why a memory leak is occurring or why WordPress doesn't offer more useful error messages. I don't call that function.. so I'm guessing it's something WordPress is doing in the background when things are run.
My info:
PHP 7.2.10
Apache 2.4
Linux Mint 19
Also happens on my server:
PHP 7.1.25
Apache 2.4
CentOS 7.6.1810
WordPress running version: 4.9.8
Share Improve this question edited Dec 11, 2018 at 16:10 treyBake asked Dec 11, 2018 at 15:57 treyBaketreyBake 12712 bronze badges 16 | Show 11 more comments2 Answers
Reset to default 1Depending on what exactly you're trying to achieve, I agree with Tom's comment, that a WP-CLI command might be better. The advantage is that the command runs from php directly on the server (usually has no max execution time, loads different php.ini, etc.) and you don't need to involve the webserver.
If that is not possible, the next best way is probably to create a custom REST endpoint. WordPress has a class WP_REST_Controller, usually I write classes that extend this and work from there. For simplicity the following example is not using inheritence, but I try to keep to the same lingo.
1. Register new route
New/custom routes are registered via register_rest_route() like so
$version = 1;
$namespace = sprintf('acme/v%u', $version);
$base = '/import';
\register_rest_route(
$namespace,
$base,
[
[
'methods' => \WP_REST_Server::CREATABLE,
// equals ['POST','PUT','PATCH']
'callback' => [$this, 'import_csv'],
'permission_callback' => [$this, 'get_import_permissions_check'],
'args' => [],
// used for OPTIONS calls, left out for simplicity's sake
],
]
);
This will create a new route that you can call via
http://www.example/wp-json/acme/v1/import/
default REST start-^ ^ ^
namespace with version-| |-base
2. Define permissions check
Maybe you need authentication? Use nonces?
public function get_import_permissions_check($request)
{
//TODO: implement
return true;
}
3. Create your actual endpoint callback
The method previously defined gets passed a WP_REST_Request object, use that to access request body, etc. To stay consistent, it is usually best to return a WP_REST_Response instead of custom printing of JSON or similar.
public function import_csv($request)
{
$data = [];
// do stuff
return new \WP_REST_Response($data, 200);
}
If you do all of this in OOP style, you'll get the following class
class Import_CSV
{
/**
* register routes for this controller
*/
public function register_routes()
{
$version = 1;
$namespace = sprintf('acme/v%u', $version);
$base = '/import';
\register_rest_route(
$namespace,
$base,
[
[
'methods' => \WP_REST_Server::CREATABLE,
'callback' => [$this, 'import_csv'],
'permission_callback' => [$this, 'get_import_permissions_check'],
'args' => [],
],
]
);
}
/**
* endpoint for POST|PUT|PATCH /acme/v1/import
*/
public function import_csv($request)
{
$data = [];
// do stuff
return new \WP_REST_Response($data, 200);
}
/**
* check if user is permitted to access the import route
*/
public function get_import_permissions_check($request)
{
//TODO: implement
return true;
}
}
But .. still 404? Yes, simply defining the class sadly doesn't work (no autoloading by default :( ), so we need to run register_routes() like so (in your plugin file)
require_once 'Import_CSV.php';
add_action('rest_api_init', function(){
$import_csv = new \Import_CSV;
$import_csv->register_routes();
});
In your functions file do something like this:
add_action('wp_ajax_import_parts_abc', 'import_parts_abc');
function import_parts_abc() {
include_once('/admin/inc/scripts/import-parts.php');
exit();
}
Then in your js file something like this:
jQuery(document).ready(function() {
jQuery('.my-button').click(function() {
jQuery.ajax({
type: 'GET',
url: ajaxurl,
data: {
action : 'import_parts_abc'
},
success: function(textStatus){
alert('processing complete')
},
error: function(MLHttpRequest, textStatus, errorThrown){
console.log(errorThrown);
}
});
});
});
.clickwhich points to this page ... never really done WP development - more of a Symfony/Magento dev - used to this way, so just assumed it would be ok haha – treyBake Commented Dec 11, 2018 at 16:11requiresome WP files. This is not the WordPress way, the documentation describes how to do AJAX or using the more modern REST API. I haven't looked too deep in your code but the thing is: you break WP flow by including some files, so it is possible that default WP behaviour is broken. Instead, if you do it the WP way, you can be sure, that things like shutdown handlers work as intended – kero Commented Dec 11, 2018 at 16:16require 'vendor/symfony/process/InputStream.php';, but create a class, make use auf autoloading, etc. -> you would use symfony architecture. And here, you should use WP architecture – kero Commented Dec 11, 2018 at 16:24php.inito give the script more memory – Tom J Nowell ♦ Commented Dec 11, 2018 at 16:33