PHP: Getting a weird issue creating tables in a database via PHP with foreach

This is a little complicated but I’ll try to keep it simple and concise.

I’m trying to have one database get updated with the information from another database every so often by just dropping all the tables and re-inserting all the information.

There are reasons why this is the best choice that don’t matter for the problem I’m having.

Somehow, my script keeps trying to create a table that it already created, and I’m not sure why.

First, here is the actual script:

<?php

try
{ // Wrapped in a try->Catch for error reporting.
	$_DB1_Gateway_PDO_Conn = new PDO
    ( // Create connection to DB1_Gateway MySQL Database.
      "mysql:host=$_DB1_Gateway_Server_Name;dbname=$_DB1_Gateway_DB_Name",
      $_DB1_Gateway_Creds['username'],
      $_DB1_Gateway_Creds['password']
    ); // Uses parsed ini file for credentials for security.
	$_DB1_Gateway_PDO_Conn->setAttribute
		( // Prepare Attribute for Error Mode Exception Handling.
			PDO::ATTR_ERRMODE, PDO::ERRMODE_EXCEPTION // Sets Error Mode to Exception
		);
	$_DB1_Gateway_PDO_Conn->setAttribute
		( // Prepare Attribute for Error Mode Exception Handling.
			PDO::ATTR_EMULATE_PREPARES, false // Disables PDO Emulation for security.
		);

	$_Select_Tables_SQL =
		("
		SHOW TABLES
		");

	// Connects to the MySQL Database and runs the Select Tables query,
	// assigning the data to the variable.
	$_Pulled_Tables = $_DB1_Gateway_PDO_Conn->prepare($_Select_Tables_SQL);
	$_Pulled_Tables->execute();

	// Sets the fetch mode to Associative Array so it doesn't return a binary.
	$_Pulled_Tables->setFetchMode(PDO::FETCH_ASSOC);

	// Fetches the entire associative array and assigns it to a variable.
	$_Fetched_Tables_Array = $_Pulled_Tables->fetchAll();

	if (count(array_filter($_Fetched_Tables_Array)) === 0)
  	{ // If the array is empty, exits and nulls the connection.
  		$_DB1_Gateway_PDO_Conn = null;
  		exit(0); // Exits with 0 error code showing no error has occurred.
  	}

	$_Table_Names = []; // Needed to be used in foreach loop below.

	foreach($_Fetched_Tables_Array as $_Table_Name)
		{
			$_Table_Names[$_Table_Name["Tables_in_gateway"]] = $_Table_Name["Tables_in_gateway"];
		}

  $_Fetched_Table_Rows_Array = []; // Needed to be used in foreach loop below.
	$_HowTo_Create_Tables_Array = [];

  foreach ($_Table_Names as $_Table_Name)
    { // Produces the multi-select query for getting all table contents.
			// Gets an array of Create Table queries for reproducing tables later.
      $$_Table_Name = // Sets variable to table name.
	      ("
	      SELECT *
	      FROM " . $_Table_Name . ";
	      "); // Creates sql variable for each table called the table.

      $_Pulled_Table_Rows = $_DB1_Gateway_PDO_Conn->prepare($$_Table_Name);
      $_Pulled_Table_Rows->execute();

			$_Fetched_Table_Rows_Array[$_Table_Name] = $_Pulled_Table_Rows->fetchAll(PDO::FETCH_ASSOC);
			// Puts array of records from table into another array
			// $key is table name.
			${'_HowTo_Create_Tables_SQL_'.$_Table_Name} =
				("
					SHOW CREATE TABLE " . $_Table_Name . ";
				");

			$_Pulled_Create_Table = $_DB1_Gateway_PDO_Conn->prepare(${'_HowTo_Create_Tables_SQL_'.$_Table_Name});
			$_Pulled_Create_Table->execute();

      $_HowTo_Create_Tables_RawArray[$_Table_Name] = $_Pulled_Create_Table->fetchAll(PDO::FETCH_ASSOC);

			$_HowTo_Create_Tables_Array[$_Table_Name] = $_HowTo_Create_Tables_RawArray[$_Table_Name][0]['Create Table'];
    }
}
catch(PDOException $_e)
{
	$_DB1_Gateway_PDO_Conn = null;
  //	echo $_e->getMessage();
	// mail_errors($_e->getMessage());
}

// Frees the result from the MySQL Database's temporary Result table.
$_DB1_Gateway_PDO_Conn = null;

try
{ // Wrapped in a try->catch for error reporting.
	$_DB2_Gateway_PDO_Conn = new PDO
		( // Create connection to DB2_Gateway MySQL Database.
		"mysql:host=$_DB2_Gateway_Server_Name;dbname=$_DB2_Gateway_DB_Name",
			$_DB2_Gateway_Creds['username'],
			$_DB2_Gateway_Creds['password']
		); // Uses parsed ini file for credentials for security.
	$_DB2_Gateway_PDO_Conn->setAttribute
		( // Prepare Attribute for Error Mode Exception Handling.
			PDO::ATTR_ERRMODE, PDO::ERRMODE_EXCEPTION // Sets Error Mode to Exception
		);
	$_DB2_Gateway_PDO_Conn->setAttribute
		( // Prepare Attribute for Error Mode Exception Handling.
			PDO::ATTR_EMULATE_PREPARES, false // Disables PDO Emulation for security.
		);

    $_DB2_Gateway_PDO_Conn->beginTransaction();

	foreach($_Fetched_Table_Rows_Array as $_Table_Name_Key => $_Table_Rows)
		{
			${'_Drop_Table_SQL_'.$_Table_Name_Key} = ("
				DROP TABLE IF EXISTS " . $_Table_Name_Key . ";
				");
			${'_Drop_Table_'.$_Table_Name_Key} =
				$_DB2_Gateway_PDO_Conn->prepare(
					${'_Drop_Table_SQL_'.$_Table_Name_Key});
			${'_Create_Table_SQL_'.$_Table_Name_Key} = $_HowTo_Create_Tables_Array[$_Table_Name];
			${'_Create_Table_'.$_Table_Name_Key} =
				$_DB2_Gateway_PDO_Conn->prepare(
					${'_Create_Table_SQL_'.$_Table_Name_Key});
			${'_Create_Table_'.$_Table_Name_Key}->execute();

	$_DB2_Gateway_PDO_Conn->commit();
}
catch(PDOException $_e)
{ // If error occurs, Exception causes automatic transaction rollback...
	$_DB2_Gateway_PDO_Conn = null; // Null connection to the database.
	// mail_errors($_e->getMessage()); // Mail errors accordingly.
}

$_DB2_Gateway_PDO_Conn = null;

?>

Now, the error I’m getting is that “user_level table already exists”. When I check the database, only that table exists. Dropping it through PHPMyAdmin’s UI doesn’t fix the issue.

Outputting the various variables and arrays to check if it is listed twice or if there’s some reason it would be recursive in the foreach has not gotten me anywhere because it is all normal AFAICT.

I just… have no idea what’s causing this issue. I’ve check syntax errors and the information being redundant but I just don’t know.

I haven’t looked at the code just yet, but from the problem that you’re trying to solve, an automated process involving mysqldump may be the better way to go about this. Do you have access to the source machine?

1 Like

No. I’ve looked into other alternatives, but this is the simplest way I’ve found.

I would be done with this already if it weren’t for this issue.

Okay, well with that out of the way…

This sounds suspiciously like one of your foreach loops isn’t cycling like you’d expect. If I understand correctly (bear with me, I haven’t used PHP in nearly a decade) it looks like your drop query and creation query are part of the same foreach loop starting at line 109. I’m having a hard time following a lot of this, so that’s about the best I’ve got.

That and a suspicion that $_Table_Name is the wrong variable name at the end of line 117. The last time $_Table_Name is referenced is in a foreach loop at 52. I would imagine that $_Table_Name is limited to that foreach scope, but if it’s possible it’s not limited to that scope, then $_Table_Name would be populated with the last table name the foreach loop cycled through.

Bah. That’s the best I’ve got for right now.

1 Like

do you expect the table schema to change? if not, there’s no reason to drop the tables. there’s probably no need to even truncate — just ON DUPLICATE KEY UPDATE.

would also recommend doing this iteratively, one table at a time, loop to copy records, repeat. especially if the db is/will grow to be of any significant size, but also would be easier to keep track of what you’re doing.

would also highly recommend not using variable variables. recommend an array instead.

is the error coming from the foreach($_Fetched_Table_Rows_Array ... loop (at ${'_Create_Table_'.$_Table_Name_Key}->execute();)? have you dumped out the generated SQL,for this, but also for the DROP statement?

Good catch. no; PHP does not have block scope (those vars are all in the same scope as surrounding code).

edit

also, i assume it’s a typo, but the code you posted is missing a } to close that last foreach loop.

3 Likes

That is it, yes.

Yes, but the odd thing is that if I have it just print the variable it’s currently on, it looks like it’s doing it correctly.

For example, say I have the following tables:

  • company
  • company_groups
  • user_levels

If I just do print_r ($_Table_Name_Key) in my foreach, it’s listing all 3 once. No repeats and no missing tables. This implies the SQL would complete as well, but it’s not because the corresponding tables aren’t being created.

I’m almost worried it’s just something to do with transactions and PDO.

Hmm, that might be it. This would result in the last table being the current result for $_Table_Name:

foreach ($_Table_Names as $_Table_Name)
    { // Produces the multi-select query for getting all table contents.
			// Gets an array of Create Table queries for reproducing tables later.
      $$_Table_Name = // Sets variable to table name.
	      ("
	      SELECT *
	      FROM " . $_Table_Name . ";
	      "); // Creates sql variable for each table called the table.

Aaaand this is almost certainly my problem:

$_HowTo_Create_Tables_Array[$_Table_Name];

I didn’t update that to $_Table_Name_Key :expressionless: I need to re-evaluate the variable names so they’re less redundant and mistakes like this are less likely.

Yes, I do. :frowning:

I’m gonna be setting this up for other databases later. Doing it this way makes it something I don’t have to touch again once I get it working for that purpose.

This isn’t a permanent thing, but at the same time, we don’t really know how long it will be before we can stop doing it. It’s in that grey area of “could be a month, could be a year”.

In the time frame I can expect this to be necessary, it will never grow large enough that this will be a problem.

So I’m not sure what you mean by this. Using PDO Prepared Statements requires a separate variable for each sql statement executed. I guess I could just string all the SQL statements together using .= since running something like SELECT * FROM table1; SELECT * FROM table2; works fine.

Yes.

I have, and it appears correct.

This is the issue. Thanks a lot guys for the quick help! :smiley: Running it now drops then creates the tables correctly.

Yeah, an error in putting it here. It’s there in my editor.

1 Like

this →$$_Table_Name is a variable variable. a variable that is named by the value of the other variable.
you never need to do this. it only makes your code harder to understand and easier to break.

as an example: you want to make some sql, named with the table name. use an array.

foreach ($table_names as $table) {
  $sql[$table] = "select * from {$table}":
}

by “one at a time” i didn’t mean to imply you couldn’t dynamically build the schema, etc… i only meant make it a one-table-at-a-time process. that is, you have steps a, b, c: right now you’re doing step a for all tables, step b for all tables, and so forth. I’m suggesting you do step a, b, c for each table individually, before moving on to the next one.

this would make it much easier to follow what your code is doing.


concept (untested). in particular, I’m relying on memory for how the sql results come back. should be mostly correct.

1 Like

:o that’s nice. Not sure why I didn’t think of using that.

I would need more than one per table however as there’s an Insert, a Drop, and a Select. It’s gonna be what it essentially is now but in an Array. I’m not sure what the functional difference is besides “It’s in an array.” ?

That looks really nice. I’m still learning so that’s why mine uses variables a lot as opposed to doing simple things in a single line.

I notice you didn’t use parameter binding. I don’t mean that as in “Hey you didn’t do a thing you should’ve” and more as “I’m curious why you didn’t use that? Is there a reason I’m missing why it’s a bad thing?”

http://php.net/manual/en/pdostatement.bindparam.php

Parameter binding afaik is best when executing a lot of the same insert queries back to back.

https://www.w3schools.com/php/php_mysql_prepared_statements.asp

I didn’t use parameter binding in all cases. Note, the INSERT query and the $fieldNames query do use parameters.

In other cases, the variable is not data (it’s the table name, or field names — identifiers), so parameterization is not possible.

Using parameters is very highly preferable, yes. you should always whenever possible. when it’s not, you need to use some other method of making sure you’re not introducing mistakes — e.g., using a whitelist of valid values.

In this case, we know that these identifiers are correct, because we’re getting them from an existing DB schema. Note the use of backticks: this is how you quote identifiers in mysql. The only risk here is that a table/field name might contain a backtick …which would be idiotic, but depending on how much you (dis)trust the source database, you might want to check for.

If you made a list of statements, as in your original code, yes. But there are other approaches! In my example, look at the main foreach loop: it handles the drop (line 28), create (30), and select(50)+insert(52) statements for each table. You don’t need to prepare these all in advance and store them (that’s what I meant by “do them one table at a time”).

Though one array of a hundred items is still better than a hundred individual variables, especially when those variables are named dynamically. Consider, for example, what would happen in your original script if one of those tables were named “_HowTo_Create_Tables_Array”.

Highly recommend you not use w3fools for anything, and especially not php or sql.

http://php.net
http://hashphp.org


1 Like

Welp, maybe I should learn 2 read.

Like I said, I’m learning. Part of my problem isn’t so much that I use dubious sources, but that finding the right sources for information is difficult with no prior information to work from.

Thanks, those are gonna help a ton.

Part of why my code is a bit much is because I am not aware of all the information and tools readily available to me. It amounts to me googling how to do something then using the example and plying it to my use case.

For example, just pulling the table names from the Information Schema, and doing so with prepared bound statements, is a great solution to my hacky Show Tables; -> foreach into array way.

Oh, a thing.

// you should define charset= in your dsn string.

What is this comment about? I’m not seeing where you define it in the example you gave.

Of course; “I’m learning” is just fine. Good, even. And I know good sources are hard to find, especially when you don’t have the knowledge/experience needed to distinguish.

That’s why I made the suggestions; it’s important to learn good things :slight_smile:

Well, I didn’t; because I don’t know what charset you’re using :wink:

“charset” defines what charset the connection should use; i.e., what character encoding your script is using. mysql uses this info to convert the data in the DB to the charset you expect, and vice-versa. In many cases, you won’t see any apparent problems from leaving it out, but in others, you can get corrupted data.

There are also obscure (but dangerous) attacks that involve sending input which tries to confuse charsets to result in sql injection, which are defeated by simply defining the charset explicitly.

In this case, you should be fine so long as you use the same charset for both connections. utf8mb4 is preferable. that would look like:

$sourceDB = new PDO(
  "mysql:host={$_DB1_Gateway_Server_Name};dbname={$_DB1_Gateway_DB_Name};charset=utf8mb4",
  ...
1 Like

one thing i missed in my example: at the last part, where you fetch records from the source DB to insert in the target DB, you want those to be fetched as an associative array (with field names as keys).

by default, PDO fetches both field names and numeric indexes, so we need to change that by setting PDO::ATTR_DEFAULT_FETCH_MODE to PDO::FETCH_ASSOC.

I made this change in the gist.

1 Like