Skip to content

Support utf8mb4 in com_localise component #286

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
andrepereiradasilva wants to merge 28 commits into from
Closed

Support utf8mb4 in com_localise component #286

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
28 commits
Select commit Hold shift + click to select a range
1d4f6f1
update table to utf8mb4 collation
andrepereiradasilva Mar 16, 2016
bc8fe9c
create table with utf8mb4 collation
andrepereiradasilva Mar 16, 2016
f772ef0
index and varchar length
andrepereiradasilva Mar 16, 2016
15b73ad
better update sequence
andrepereiradasilva Mar 16, 2016
d0fb14c
unicode, not general
andrepereiradasilva Mar 16, 2016
ad947a0
add the new #__localise_revised_values table also in utf8mb4 charset …
andrepereiradasilva Mar 16, 2016
0fb4ff5
Convert the new table #__localise_revised_values to utf8mb4 too
andrepereiradasilva Mar 16, 2016
a5a744f
remove comments that don't make sense is this scenario
andrepereiradasilva Mar 16, 2016
dd9a871
change path collation to utf8mb4_bin on update
andrepereiradasilva Mar 16, 2016
f60eb2a
remove language table comment
andrepereiradasilva Mar 16, 2016
795b3bd
create path with utf8mb4_bin collation
andrepereiradasilva Mar 16, 2016
9792363
solve conflicts
andrepereiradasilva Mar 17, 2016
b4f8aeb
Remove unique key and resize idx_path index to 191
andrepereiradasilva Mar 18, 2016
468d125
Remove unique key and resize idx_path index to 191
andrepereiradasilva Mar 18, 2016
15124ce
cleaner comments
andrepereiradasilva Mar 18, 2016
fbdf492
PoC install over utf8 and utf8mb4 db systems
andrepereiradasilva Mar 19, 2016
e144b54
correct update utf8mb4
andrepereiradasilva Mar 19, 2016
9edc11c
corrections
andrepereiradasilva Mar 19, 2016
839c91b
4.0.15
andrepereiradasilva Mar 19, 2016
c865303
explain in comments how the utf8/utf8mb4 sql scripts runs
andrepereiradasilva Mar 19, 2016
d471500
optimize comments
andrepereiradasilva Mar 19, 2016
99e49b5
file, not schemepath
andrepereiradasilva Mar 19, 2016
6df0373
better comment
andrepereiradasilva Mar 19, 2016
fd79cb3
copy/paste type
andrepereiradasilva Mar 19, 2016
aa7c90f
solve conflicts after latest merge
andrepereiradasilva Mar 20, 2016
ac3ba86
improve comments and change year to 2016
andrepereiradasilva Mar 20, 2016
b40932b
adjust
andrepereiradasilva Mar 20, 2016
71c1e02
use dates in sql update files like the core does
andrepereiradasilva Mar 20, 2016
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 22 additions & 0 deletions component/admin/sql/install/mysql/install.utf8mb4.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
CREATE TABLE IF NOT EXISTS `#__localise` (
`id` int(11) unsigned NOT NULL AUTO_INCREMENT,
`asset_id` int(11) NOT NULL,
`path` varchar(400) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL DEFAULT '',
`checked_out` int(10) unsigned NOT NULL DEFAULT '0',
`checked_out_time` datetime NOT NULL DEFAULT '0000-00-00 00:00:00',
PRIMARY KEY (`id`),
KEY `idx_path` (`path`(191))
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 DEFAULT COLLATE=utf8mb4_unicode_ci;

CREATE TABLE IF NOT EXISTS `#__localise_revised_values` (
`id` int(11) unsigned NOT NULL AUTO_INCREMENT,
`client` varchar(255) NOT NULL,
`reftag` varchar(6) NOT NULL,
`tag` varchar(6) NOT NULL,
`filename` varchar(255) NOT NULL,
`revised` tinyint(1) NOT NULL DEFAULT 0,
`key` varchar(255) NOT NULL,
`target_text` mediumtext NOT NULL,
`source_text` mediumtext NOT NULL,
PRIMARY KEY (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 DEFAULT COLLATE=utf8mb4_unicode_ci;
Empty file added component/admin/sql/updates/mysql-utf8mb4/0.0.1.sql
View file
Open in desktop
Empty file.
0 component/admin/sql/updates/mysql/4.0.15.sql → ...dates/mysql-utf8mb4/4.0.15-2016-03-16.sql
View file
Open in desktop
File renamed without changes.
21 changes: 21 additions & 0 deletions component/admin/sql/updates/mysql-utf8mb4/4.0.15-2016-03-20.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
-- Step 1 of the UTF-8 Multibyte (utf8mb4) conversion for MySQL.
-- Drop indexes which will be added again in step 2.
ALTER TABLE `#__localise` DROP KEY `idx_path`;

-- Step 2 of the UTF-8 Multibyte (utf8mb4) conversion for MySQL.
-- Step 2.1: Limit indexes so their max allowed lengths would not get exceeded with utf8mb4.
ALTER TABLE `#__localise` ADD KEY `idx_path` (`path`(191));

-- Step 2.2: Enlarge columns to avoid data loss on later conversion to utf8mb4.
ALTER TABLE `#__localise` MODIFY `path` varchar(400) NOT NULL DEFAULT '';

-- Step 2.3: Convert all tables to utf8mb4 chracter set with utf8mb4_unicode_ci collation.
ALTER TABLE `#__localise` CONVERT TO CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
ALTER TABLE `#__localise_revised_values` CONVERT TO CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;

-- Step 2.4: Set collation to utf8mb4_bin for formerly utf8_bin collated columns.
ALTER TABLE `#__localise` MODIFY `path` varchar(400) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL DEFAULT '';

-- Step 2.5: Set default character set and collation for all tables.
ALTER TABLE `#__localise` DEFAULT CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
ALTER TABLE `#__localise_revised_values` DEFAULT CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
1 change: 1 addition & 0 deletions component/admin/sql/updates/mysql-utf8mb4/index.html
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
<!DOCTYPE html><title></title>
12 changes: 12 additions & 0 deletions component/admin/sql/updates/mysql/4.0.15-2016-03-16.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
CREATE TABLE IF NOT EXISTS `#__localise_revised_values` (
`id` int(11) unsigned NOT NULL AUTO_INCREMENT,
`client` varchar(255) NOT NULL,
`reftag` varchar(6) NOT NULL,
`tag` varchar(6) NOT NULL,
`filename` varchar(255) NOT NULL,
`revised` tinyint(1) NOT NULL DEFAULT 0,
`key` varchar(255) NOT NULL,
`target_text` mediumtext NOT NULL,
`source_text` mediumtext NOT NULL,
PRIMARY KEY (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
16 changes: 14 additions & 2 deletions localise.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,26 +6,38 @@
<author>Christophe Demko</author>
<author>Jean-Marie Simonet</author>
<author>Ifan Evans</author>
<copyright>(C) 2015 Open Source Matters. All rights reserved.</copyright>
<copyright>(C) 2016 Open Source Matters. All rights reserved.</copyright>
<authorEmail>admin@joomla.org</authorEmail>
<authorUrl>https://github.com/joomla-projects/com_localise</authorUrl>
<version>4.0.14-dev</version>
<version>4.0.15-dev</version>
<license>http://www.gnu.org/licenses/gpl-2.0.html GNU/GPL</license>
<description>COM_LOCALISE_XML_DESCRIPTION</description>
<scriptfile>install.php</scriptfile>
<install>
<sql>
<!--
Pre Joomla 3.5.0 doesn't recognize utf8mb4 charsets so they will only run the utf8 charset script.
Joomla 3.5.0+ versions will run the utf8mb4 sql file on utf8mb4 capable database servers and run the utf8 sql file on non utf8mb4 capable database servers.
-->
<file driver="mysql" charset="utf8">sql/install/mysql/install.sql</file>
<file driver="mysql" charset="utf8mb4">sql/install/mysql/install.utf8mb4.sql</file>
</sql>
</install>
<uninstall>
<sql>
<!-- Since it just drops tables, the uninstall sql file can be the same for utf8mb4 capable database servers or non utf8mb4 capable database servers. -->
<file driver="mysql" charset="utf8">sql/install/mysql/uninstall.sql</file>
</sql>
</uninstall>
<update>
<schemas>
<!--
The order here is important.
Pre Joomla 3.5.0 will only run the update sql files in the first schemapath, because pre Joomla 3.5.0 versions don't check any charset attribute, instead they take the first one they find for the chosen database server type.
Joomla 3.5.0+ versions will run the update sql files in the utf8mb4 schemapath on utf8mb4 capable database servers and the update sql files in the utf8 schemapath on non utf8mb4 capable database servers.
-->
<schemapath type="mysql" charset="utf8">sql/updates/mysql</schemapath>
<schemapath type="mysql" charset="utf8mb4">sql/updates/mysql-utf8mb4</schemapath>
</schemas>
</update>
<media destination="com_localise" folder="media/com_localise">
Expand Down