Apparently there are items and plugins out there that still use legacy
color codes within text components, and which thereby break this
heuristic.
Our remaining approach to differentiate between legacy (plain) and
modern (JSON-based) text is to check if a particular text can be parsed
as JSON-based text. This approach is not perfect either as there are
ambiguous cases that it cannot resolve correctly. However, these cases
are hopefully rare enough in practice that this approach remains
suitable.
For example, this affects empty Strings, and Strings that consist only
of whitespace. In lenient mode this would also affect text that contains
comments, but that is not the case currently.
Minecraft's component parser (i.e. Gson) produces null for these texts
instead of throwing a parse exception. By interpreting this text as
legacy text we correctly create text components for them that contain
their contents.
Issues resolved by this:
* SPIGOT-5063: Internal text representation of ItemStacks changes during ItemStack serialization. This issue was initially primarily concerned with the conversion between color text attributes to legacy color codes.
* SPIGOT-5304: Internal text representation of ItemStacks changes when opening the inventory (in creative mode). In particularly, this issue is also concerned with the conversion between plain text representations to non-plain ones.
* SPIGOT-5656: Internal text representation of ItemStacks changes during ItemStack serialization. This issue is particularly concerned with reordering of text attributes in the text's Json representation.
* SPIGOT-3206: Internal text representation of book pages changes during ItemStack serialization.
* SPIGOT-5350: Any non-plain text features are stripped from books during ItemStack serialization.
* SPIGOT-5980: Written books are marked as 'resolved' during ItemStack serialization and on various inventory interactions, even though they aren't, and thereby breaking any non-resolved page contents.
* SPIGOT-4672: Since item display names are serialized in their internal Json representation, any translatable components get properly persisted as well now.
---------
Minecraft uses text components to represent text. Internally Minecraft stores these components as Json formatted Strings and dynamically parses the text components from this Json representation whenever required.
In some cases Minecraft will create the text components and then convert them to Json itself for the internal storage. In other cases the Json representation is specified by users (eg. in Minecraft give commands, loot tables, mob equipment specified via Minecraft's summon commands, etc.).
There are many different ways in which the same text components can be represented in Json. When Minecraft compares objects which store this textual information, it takes the exact Json representation into account to determine whether the objects are considered equal. For example, ItemStacks will not match (and therefore not stack) if there is a difference in this internal Json representation for at least one if the item's text attributes (such as display name, lore, book pages, etc.). And when specifying nbt data in command selectors (eg. to only match entities/players which hold an item with specific name), the selector compares the raw Json representation as well.
As long as the Json representation is valid and can be parsed, Minecraft will not modify or normalize it.
However, under various circumstances Spigot converts this text information from the internal Json representation to text components (and in some cases even to plain text with legacy color codes) and then later tries to convert the text from these representations back to text components in the Json representation. Because this backwards conversion is in many cases not able to reproduce the original Json representation, the internal data of some affected Minecraft objects (ItemStacks, TileEntities, Entities, etc.) will in some cases get modified.
One especially notable situation in which this issue can come up is Bukkit's configuration serialization of ItemStacks: When a plugin serializes and later deserializes ItemStacks with display name, localized name, lore, or book pages of signed books, Spigot would convert these textual ItemStack attributes to plain text with legacy color codes and later try to convert those back to chat components in the Json representation. If the reconstructed Json representation does not match the original representation, the deserialized ItemStacks would no longer match nor stack with any original ItemStacks.
This case is particularly common if the original ItemStacks are created by users via some vanilla Minecraft mechanism (eg. Minecraft's give command, loot tables, mob equipment specified via Minecraft's summon command, etc.) and the used internal text representation for the created ItemStacks does not match the text representation produced by Spigot. This is also quite likely to be case, because the internal text representation produced by Spigot can sometimes be slightly verbose and, until recently, contained legacy color codes which cannot be used in Minecraft commands in-game.
However, this issue is not limited to items created by users, but affects items created by Minecraft itself as well.
Other cases in which Spigot itself (without any plugins involved) will convert between these text representations include dragging items around inside the inventory or opening the inventory while in creative mode. In these cases Spigot creates Bukkit representations of the affected items for use in Bukkit events and then, after the events have been handled, converts these Bukkit representations back to Minecraft items. See for example SPIGOT-5656 and SPIGOT-5304.
The idea of these changes is to avoid this back and forth conversion between the internal Json representation and the text component or plain text representations in various situations in which it is not actually required:
* CraftMetaItem stores the raw original Json representation for the display name, localized name, lore and pages of signed books now. As long as no plugin modifies these text attributes via the API, they can be reapplied in their original form to an ItemStack.
* The configuration serialization will serialize the original Json representation for these text attributes now so that it can also be restored during deserialization.
* However, in order to still be able to deserialize previously serialized items, and in order to allow users to specify text in the more simple plain representation in configuration files, we also still accept plain text during deserialization. Our approach is to check if the serialized text contains legacy color codes, in which case we convert it to chat components using our own converter and then to Json. Otherwise we try to parse it via Minecraft's Json parser. If the parsing fails due to the text not being valid Json, we interpret the text as plain text and convert it via our own converter as well.
* Various duplicated code has been removed from CraftMetaBookSigned and instead the base CraftMetaBook class allows sub classes to override the relevant aspects of how pages are parsed, serialized and deserialized.
* The BlockStates for command blocks and signs preemptively retrieved the custom name and sign line components, converted them to plain text and later converted them back to text components when applying the BlockState. We now only perform this conversion if a plugin has explicitly modified these texts.
Other changes:
* Minor: We also retrieve, convert and update a few other BlockState attributes directly from the underlying snapshot and only when requested by plugins now.
* SPIGOT-5980: Written books did not properly persist their 'resolved' attribute, resulting in unresolved book pages not getting resolved.
* There are methods to get and set the resolved value for books. However, these are not yet exposed in Bukkit.
* Minor fix: CraftMetaBook#isBookEmpty did not check some of the book attributes. This is probably a minor issue, but for consistency reasons there are checks for the missing attribute(s) now.
----
Covered cases
---
* By remembering the raw original String data, we can persist the exact text representation (eg. the ordering of elements within the Json text object (SPIGOT-5656), used style of escaping quotes (single quotes, escaped double quotes, etc.), use of plain texts (SPIGOT-5304), used boolean style, modern text component features such as translatable texts (SPIGOT-4672), etc.). All of these differences would otherwise cause the ItemStack to no longer be considered equal to its original.
* An empty String in the serialized config data results in no display name rather than an empty display name, like before. An item with explicitly empty display name (`{display: {Name: '""'}}`) is saved as `'""'` and can also be loaded from that representation again.
* Any plain texts, with or without color codes, which don't parse as Json (eg. `display-name: 'Bla'`) are still getting run through Spigot's text to components converter, like before.
* We can now also persist empty but explicitly present lore (`{display:{Lore:[]}}`). Previously this would get removed when the ItemMeta gets reapplied to the item. And ItemMeta#equals would return true for items with and without this empty lore data, even though Minecraft considers them to be different. For plugins using the API there should be no change: #hasLore still checks whether the lore is both present and not empty, and #getLore returns an empty list instead of null in this case (however, it previously already returned an empty list in this case). And setting the lore to an empty list via #setLore will still result in an item with no lore.
* Similarly, we can also persist explicitly specified but empty lists of book pages now.
----
Cases that are not covered (i.e. which may lead to changes in items), but were already not covered previously:
----
* NBT data for text that is not actually of type String.
* Empty or unexpected entries within the display compound.
* Variations in the NBT data representation in item features other than the above mentioned ones.
* Texts containing color codes. During deserialization these texts get interpreted as plain text and converted to a text component representation. This will break the serialization of any ItemStacks which actually use a text component representation with embedded color codes for some reason. Usually the likelihood for encountering such items in practice would probably be small. However, in the past (pre MC 1.16) Spigot would actually produce such items during ItemStack deserialization or when plugins created ItemStack via the Bukkit API. However, Spigot has changed the text representation it produces in MC 1.16, so any previously created and still existing items with this text representation are already problematic anyways now. See SPIGOT-5964. A fix for this linked issue (eg. the automatic conversion of these items) would probably resolve this deficit here as well.
* Spigot's String to text components converter produces quite verbose components since 1.16. See SPIGOT-5964 as well. However, this applies regardless of the changes of this PR.
* Book ItemStacks with more pages than 100 pages or oversized pages are truncated (like before) and may therefore change.
hange.
