Editing Unicode (section)

== Managing 32-Bit Unicode Characters ==

In recent years there has been a massive expansion to the number of Unicode characters, to the extent that there is not enough bits to represent each character in UTF-8.  Therefore there was an augmentation called UTF-8 MB4, which allows this extra Unicode characters, such as 𝒪 (mathematical script capital O).  If you need support for such characters there are two things you need to do:

1. Modify '''perl_lib/EPrints/Database/mysql.pm''' and chnage the line:
 $self->do("SET NAMES 'utf8'");
to:
 $self->do("SET NAMES 'utf8mb4'");

2. Modify the column in the table.  You cannot do this for VARCHAR columns unless they are 191 characters or fewer.  Due to the maximum number of bytes allowed for a VARCHAR column.
 ALTER [table] MODIFY [column] TEXT CHARACTER SET utf8mb4;
 ALTER [table] MODIFY [column] LONGTEXT CHARACTER SET utf8mb4;
 ALTER [table] MODIFY [column] VARCHAR(191) CHARACTER SET utf8mb4;

There are plans to implement these changes in [[EPrints 3.4.4]], so that newly created repository will be able to handle 32-bit Unicode characters.  If this is implemented upgrading to 3.4.4 will likely still require manually updating existing fields.

For more information see https://www.eversql.com/mysql-utf8-vs-utf8mb4-whats-the-difference-between-utf8-and-utf8mb4/