=head1 UPGRADING FROM RT 4.2.0 and greater

The 4.4 release is a major upgrade and as such there are more changes
than in a minor bugfix release (e.g., 4.2.2 to 4.2.3) and some of these
changes are backward-incompatible. The following lists some of the notable
changes, especially those that might require you to change a configuration
option or other setting due to a change in RT. Read this section carefully
before you upgrade and look for changes to features you currently use.

See F<devel/docs/UPGRADING-4.4> for internals changes relevant to
extension writers.

=over

=item *

RT now natively supports external authentication systems like
LDAP and custom database setups. For new users wishing to use this
functionality, see the L<RT::Authen::ExternalAuth::LDAP> or
L<RT::Authen::ExternalAuth::DBI> documentation.

Users of the existing
L<RT::Authen::ExternalAuth|https://metacpan.org/pod/RT::Authen::ExternalAuth>
extension should remove C<RT::Authen::ExternalAuth> from the plugins list.
Please also remove F<RT-Authen-ExternalAuth> from your RT
installation.

=item *

L<RT::Extension::LDAPImport|https://metacpan.org/pod/RT::Extension::LDAPImport>
has been moved into core RT.

Users of the existing LDAPImport extension should remove
C<RT::Extension::LDAPImport> from the plugins list. Please adjust any
cronjobs or external scripts which invoke
F<rtldapimport> to instead
invoke F</usr/sbin/rt-ldapimport-5>. Please also remove
F<RT-Extension-LDAPImport> from your RT installation.

=item *

As of 4.4, the C<RT::Extension::Assets> functionality has been moved to
core RT. New and upgrading users will automatically get all Assets
functionality. Assets documentation can be found at F<docs/assets.pod>.
A tutorial on customizing your Assets install can be found at
F<docs/customizing/assets/tutorial.pod>.

To cleanly hide it from upgrading RT installs, the Assets menu is controlled
by the right ShowAssetsMenu. This right is only grantable globally to groups
or users. If you were previously using the Assets extension, you can grant the
ShowAssetsMenu right to Privileged or another role or group to make the menu
visible.

Users who are currently using the
L<RT::Extension::Assets|https://bestpractical.com/assets/> extension
should remove C<RT::Extension::Assets> from the plugin list and run the
F</usr/share/request-tracker5/etc/upgrade/upgrade-assets> utility after completing all the other
upgrade steps from the F<README>. Please also remove
F<RT-Extension-Assets> from your RT installation.

=item *

As of 4.4, SLA tracking is also a core feature, so C<SLA> is a core field
on tickets for queues on which SLA tracking is enabled. Please see the SLA
section in F<RT_Config.pm> as well as F<docs/customizing/sla.pod> for
details on configuring RT's SLA feature.

Users currently using
L<RT::Extension::SLA|https://metacpan.org/pod/RT::Extension::SLA> should
do the following to migrate to core SLA functionality after running
the main RT code and database upgrade steps successfully:

=over 4

=item * Remove C<RT::Extension::SLA> from your plugin list in C<RT_SiteConfig.pm>

=item * Run the upgrade script F</usr/share/request-tracker5/etc/upgrade/upgrade-sla>

=item * Update the format of your C<%ServiceAgreements> configuration

You can keep the same configuration, but it is now set with a C<Set>
call like all standard RT features, the leading C<RT::> is removed,
and the C<=> becomes a C<,>. You can
see examples in F<docs/customizing/sla.pod>.

=item * Update the format of your C<%ServiceBusinessHours> configuration

If you have a Business Hours configuration, update your configuration
in C<RT_SiteConfig.pm> with the same changes as described above for
C<%ServiceAgreements>.

=item * (Optional) Remove the directory F<RT-Extension-SLA>

You can remove this directory and all of its contents from your RT
installation to uninstall the previous extension code

=back

Note that since SLA is now a core ticket value, it is currently not possible
to have different levels per queue as was previously possible when using
multiple SLA custom fields. Currently all queues share the same set of levels
defined in C<%ServiceAgreements>.

=item *

RT can now natively store attachments outside the database, either on disk, in
Dropbox, or on Amazon S3. For more information see L<RT::ExternalStorage> and
C<%ExternalStorage> in F<RT_Config.pm>.

Users who are currently using
L<RT::Extension::ExternalStorage|https://metacpan.org/pod/RT::Extension::ExternalStorage>
should remove it from the plugin list. Please adjust any cronjobs or external
scripts which invoke
F<local/plugins/RT-Extension-ExternalStorage/bin/extract-attachments>
to instead invoke F</usr/sbin/rt-externalize-attachments-5>. Please also remove
F<RT-Extension-ExternalStorage> from your RT installation.

=item *

RT now has the functionality from L<RT::Extension::ParentTimeWorked> built in,
with some modifications. In the ParentTimeWorked extension, when a child ticket's
TimeWorked field was updated, the parent ticket's TimeWorked field was also
incremented. Starting with RT 4.4.2, the time from child tickets is available
in the Total Time Worked field and the Time Worked value for the parent ticket
is independent.

RT now also has built-in functionality to show time worked per user.

For RT 4.4.0 and 4.4.1, these functions were enabled via scrips.
New installs had these scrips by default. Upgrades got them too, but set as
disabled so they didn't interfere with your existing configuration. These
functions are now dynamic and no longer require these scrips.

See L</"UPGRADING FROM 4.4.1 AND EARLIER"> for additional details on the changes
introduced in 4.4.2.

Users who are currently using
L<RT::Extension::ParentTimeWorked|https://metacpan.org/pod/RT::Extension::ParentTimeWorked>
should remove it from the plugin list. Please also remove
F<RT-Extension-ParentTimeWorked> from your RT installation.

=item *

The support for C<jsmin> (via the C<$JSMinPath> configuration) has been
removed in favor of a built-in solution.

=item *

You can now split settings from F<RT_SiteConfig.pm> into separate files under
an F</etc/request-tracker5/RT_SiteConfig.d/> directory. All files ending in C<.pm> will be
parsed, in alphabetical order, after the main F<RT_SiteConfig.pm> is loaded.

You also no longer need the C<1;> at the end of site config files.

=item *

When creating linked tickets from the Links box, Time Worked, Time
Estimated, and Time Left will no longer be copied.  This simplifies time
reporting.

=item *

Custom fields with categories will be split out into hierarchical custom
fields.

=item *

Homepage component "Quicksearch" has been renamed to "QueueList" to reflect
what it actually is. Please update C<$HomepageComponents> accordingly if you
customized it in F<RT_SiteConfig.pm>.

=item *

RT::Interface::Email no longer exports functions.

=item *

Incoming email now always creates users for the C<From:> address.
Previously, if the C<CreateTicket> right was not granted to C<Everyone>
or C<Unprivileged>, the email would be rejected without a user; now, the
user will be created even when the mail is rejected.

=item *

The L<RT_Config/@MailPlugins> functionality has been rewritten; mail
plugins written for previous versions of RT will not function as
expected.  See F<docs/extending/mail_plugins.pod>

=item *

The C<$UnsafeEmailCommands> option has been replaced with two mail
plugins, L<RT::Interface::Email::Action::Take> and
L<RT::Interface::Email::Action::Resolve>.

=item *

The C<RejectOnUnencrypted> option to L<RT_Config/%Crypt> has been
replaced with a mail plugin,
L<RT::Interface::Email::Authz::RequireEncrypted>.

=item *

We added L<DateTime::Format::Natural> support to parse user-entered datetimes,
which will be called only when L<Time::ParseDate> fails to parse. You can tell
RT to prefer L<DateTime::Format::Natural> in F<RT_SiteConfig.pm>:

    Set($PreferDateTimeFormatNatural, 1);

You might do this if you prefer its handling of some syntax (e.g. "last
month").

=item *

The "Display Columns" section in the Search Builder has been ajaxified, so you
don't need to reload the whole page to adjust formatting.

=item *

When corresponding/commenting on tickets, you can include attachments that
already exist on the ticket so you don't need to upload them again.

This functionality was previously provided by the third-party
L<RT::Extension::AddAttachmentsFromTransactions>.

=item *

We added a C<Disabled> column to the Articles table, so now instead of deleting
articles, we disable them, like other RT records. This helps maintain RT's
auditability goals.

For new deploys we also establish some common defaults for Articles: a General
class and a Content textarea custom field.

=item *

We removed the C<InitialPriority>, C<FinalPriority>, and C<DefaultDueIn>
columns from the Queues table. In their stead we have a more general
C<< ->DefaultValue >> call, e.g.

    $queue->DefaultValue('InitialPriority')
    $queue->DefaultValue('FinalPriority')
    $queue->DefaultValue('Due')

Note that "Due" can now be anything that can be parsed as a date.

With this, we've also added the ability to add default values for "Starts" and
custom fields. All of them may be set on a queue's DefaultValues admin page.

=item *

We've added a new config option and preference for C<HideUnsetFieldsOnDisplay>
to hide unset fields on the ticket display page. Note that this option does not
affect custom fields. If you would like to hide custom fields too, please see
L<RT::Extension::CustomField::HideEmptyValues>.

=item *

We've added another option for rendering ticket history: infinite scroll. This
loads several items at a time as you scroll down the page. This should improve
the perceived performance of ticket display. Users can enable this option by
changing the "Show history" preference to "as you scroll". You can also change
the system default in F<RT_SiteConfig.pm> just like the existing options with

    Set($ShowHistory, 'scroll');

=item *

RT now ships with keyboard shortcuts, both global (e.g. C</> to focus on quick
search) and page-specific (e.g. C<x> to toggle a checkbox in bulk update).
Press C<?> in your browser to see what shortcuts can be used on the current
page.

=item *

We added a modern file uploader (Dropzone, L<http://www.dropzonejs.com>). Now
it's much easier to upload multiple files at a time, by dragging them into the
page.

The legacy attachment is still available when the modern uploader fails to
load.

=item *

We added two meta-statuses to TicketSQL: C<__Active__> and C<__Inactive__>, so
you no longer need to explicitly list all the active or inactive statuses like:

    Status = 'new' or Status = 'open' or Status = 'stalled'

Now you can simply write:

    Status = '__Active__'

=item *

We added an "Entry Hint" to custom fields. Previously, we used the custom
field's C<FriendlyType> as its hint, but now it is replaced by the per-CF
customizable C<EntryHint> field.

=item *

RT now has support for custom roles, along the lines of Requestor, Owner, Cc,
and AdminCc. You can select whether a custom role may have only a single member
or multiple members (though you cannot change the role's arity after it's been
created). You can assign custom roles to queues much like custom fields. Rights
may be granted on custom roles globally or per-queue, and they may be searched
in TicketSQL and the search builder with syntax like the following:

    CustomRole.{Engineer}.EmailAddress LIKE 'bestpractical.com'
    CustomRole.{3} = '__CurrentUser__'

Finally you may notify custom roles in scrips by creating a
L<RT::Action::Notify> with an C<Argument> containing the role name or
C<RT::CustomRole-#>. By default, custom roles will be added as Cc on outgoing
mail but you can set them as To or Bcc by using a new slash syntax:

    AdminCc, Engineer
    Owner, RT::CustomRole-3/To, Cc
    Engineer/To, Sales/Bcc

Custom role members can be added at the queue level (for multi-member roles) as
well as on individual tickets (for both single-member and multi-member roles).

=item *

Users can now select which days of the week they'd like Daily dashboard
subscriptions to be mailed to them. This removes the now-duplicate C<M-F>
subscription type and makes the C<Daily> subscription type respect new per-day
checkboxes.

=item *

We've dropped several never-used columns on the C<Users> and C<Tickets> tables
and several columns that haven't been used in a very long time. The first 4.4
upgrade step makes sure there is no content in these fields, just in case an
extension or local customization had appropriated any of them.

For Users, we've dropped C<EmailEncoding>, C<WebEncoding>,
C<ExternalContactInfoId>, C<ContactInfoSystem>, C<ExternalAuthId>,
C<AuthSystem>, and C<PGPKey>. (Note: GPG keys have always been stored in
attributes on the User record in RT, never in the C<PGPKey> column)

For Tickets, we've dropped C<IssueStatement>, C<Resolution>, and C<Disabled>.

=item *

Users now receive a warning message when they type the word "attach" without
having attached anything to the ticket, much like many mail clients offer. This
should help cut down on the "oops, I forgot the attachment" followup mails.

=item *

We've upgraded our version of jQuery from 1.9.1 to 1.11.3 and jQuery UI from
1.10.0 to 1.11.4.

https://jqueryui.com/upgrade-guide/1.10/
https://jqueryui.com/upgrade-guide/1.11/

=item *

We've upgraded our version of CKEditor from 4.0.1 to 4.5.3.

=item *

In rights management pages, sections with rights assigned now have a gray
callout.

=item *

We've removed the C<Type> column for Groups, which was redundant. C<Domain>
identifies the type of group.

=item *

We've removed the C<ObjectId> for Principals, which was redundant, as it should
always be the same as C<id>.

=item *

We've added a new config option C<%ReferrerComponents> to customize how
referrer whitelists and blacklists work.

=item *

We now place tmpfiles generated during email parsing under /tmp, which
should allow the system tmp-cleaner to purge any dangling files, and
eases configuration of RT under SELinux.

=item *

Squelching now applies to all updates in the request, instead of only the
initial correspond/comment transaction.

=item *

We removed the unused "on merge" lifecycle property, as merged tickets cannot
be found via searches or charts.

=item *

We now allow ModifyTicket to change nobody to someone else, without OwnTicket.

Previously, we required OwnTicket if the ticket was currently unowned, no
matter if you were taking it yourself or assigning it to someone else.

We now allow nobody -> self with OwnTicket and (ModifyTicket, ReassignTicket,
or TakeTicket), and nobody -> someone else with ModifyTicket or ReassignTicket.
This also closes a bug where TakeTicket and OwnTicket was enough to change the
owner from Nobody to someone else.

=item *

We now no longer large round number of hours worked into days, since "7 days
worked" may imply only 56 business hours instead of 168 real hours.

=item *

We've added a ticket timer feature for automatically keeping track of time.
This feature pops open a small JavaScript window with a very much trimmed down
RT interface.

=item *

On ticket display pages we now make the queue name a hyperlink to a search for
all active tickets in that queue.

=item *

There is a new widget on the ticket display page that allows you to create a
ticket in a different queue that will automatically be linked to the given
ticket. In the Links section on the display page, choose the type of link and
the queue, and click Create. The create ticket form will be opened. When the
ticket is created, it will be linked to the previous ticket.

=item *

IPv6 custom field values are now displayed in compressed form, for clarity.
Searching and updating handle both representations as you'd expect.

=back

=head1 UPGRADING FROM 4.4.0 AND EARLIER

=over 4

=item *

The Dashboard subscription recipient options have been greatly expanded from
a single text field (which happened to support multiple email address
separated with a comma) to a robust user/group search. There is a database
upgrade step for migrating the old "Recipient" textbox values.

=item *

In versions of RT before 4.4.1, under certain conditions Saved Charts were
not completely saved to the database, which led to them not showing up when
added to dashboards. There is a database upgrade step that fixes such charts
so they will now show up on dashboards, and of course the underlying bug has
been corrected.

=item *

We updated default RT searches that previously specified 'new' and 'open'
(and sometimes, 'stalled') status to use the new Status='__Active__' and
Status='__Inactive__' syntax where appropriate. For new RT installs, this
means the default RT at a Glance searches will now work with custom
lifecycles without updating these searches.

For existing RT sites, you may want to update the "highest priority tickets
I own" and 'newest unowned tickets' system searches to use
Status='__Active__' rather than specific statuses.

=item *

Users may now select a specific language for dashboard email subscriptions.
Administrators can customize the method by which dashboard email language is
chosen (including specifying an ultimate fallback other than English) with
the @EmailDashboardLanguageOrder RT_Config option.

This may change which language users receive their dashboards in. To fix
this, users may need to specify their language explicitly on their "About
Me" page. Administrators may also want to set up a default language that
matches their userbase better than RT's default of English.

=item *

The "hide unset fields" preference now also hides unset custom fields,
obsoleting RT::Extension::CustomField::HideEmptyValues. The implementation
was also changed from omitting HTML tags for such fields to adding an
`.unset` class name and hiding such fields with CSS.

=item *

Long attachment lists now show only the five newest attachments with an AJAX
"Show all" link. This should improve the performance and usability of both
ticket display and ticket reply pages.

=item *

rt-crontool used with multiple actions may have been sending email to the
wrong recipients; we've fixed this issue (by reloading the template object
for each action) so please double check that such cronjobs are still sending
mail to the right people.

=item *

We now explicitly depend on the Pod::Select Perl module since it was removed
from the Perl core starting in 5.18. If you're on a recent version of Perl
you will most likely need to install this dependency.

This is installed on Debian when installing the request-tracker5 package.

=item *

We now automatically enable ExternalAuth when the ExternalSettings config is
declared, so you may remove Set($ExternalAuth, 1); from your config.

=item *

We've deprecated the /Admin/CustomFields/Modify.html callback named
AfterUpdateCustomFieldValue. It will be removed in RT 5.0.

=back

=head1 UPGRADING FROM 4.4.1 AND EARLIER

=over 4

=item *

Changes to Time Worked Handling

As noted above, in RT 4.4 we pulled in some time tracking features from
extensions, specifically adding time worked on child tickets to the
parent and showing time worked per user. In RT 4.4.2, we made some important
changes to this functionality.

The new features were installed but disabled for upgrades, so these changes
are most important to RT users who installed RT starting with version 4.4.0
or 4.4.1, or who users who upgraded from a previous RT and then manually
enabled the time tracking scrips.

=over 4

=item *

Parent Time Worked

Based on feedback from users and our own experience, we determined that
updating the Time Worked value on the parent ticket made it difficult
to differentiate time recorded from children tickets from time recorded
directly on the parent ticket. To clarify the history for these time
entries, we created a new dynamic value called Total Time Worked that
is a sum of Time Worked from the parent and all child tickets. You can
activate this feature with the C<$DisplayTotalTimeWorked> configuration
option in C<RT_SiteConfig.pm>.

We generate this when displaying the ticket and we no longer update Time
Worked on the parent ticket. This change also has the advantage that Total
Time Worked automatically updates when a child ticket is added or removed.
Total Time Worked is also available as a column for reports generated with
the Query Builder.

B<Important:> If you were previously using the parent time worked feature,
the Time Worked value on parent tickets already contains time from child
tickets. Total Time Worked will therefore be incorrect since it will add
up all children then add the parent time (which already has the
child time). You can choose to keep using the scrip and not show Total Time
Worked, or update Time Worked on the parent ticket to have just parent time
worked.

Contact Best Practical at contact@bestpractical.com if you have questions
about these changes.

=item *

Time Worked by User

For the user time summary feature, we found some odd cases where a time
value would not get recorded (if the scrip was mistakenly disabled, for
example). In these cases, since the data was stored internally, it was
difficult to update the user time values. We have made this feature
dynamic so the user time information for the current ticket is generated
when the ticket is displayed.

If you previously used this feature, the user time data stored in an
attribute on each ticket will still be in the database. Since it is only
visible directly in the database, you can safely ignore it. However, if
you want to remove it, you can locate these records with the following:

    SELECT id, Name FROM Attributes WHERE ObjectType = 'RT::Ticket' AND Name = 'TimeWorked';

You can delete these records from the Attributes table using a DELETE
and the same WHERE clause. Always make sure you have a known good backup
of your database before making updates.

=back

=item *

RT now has the functionality from
L<RT::Extension::AdminConditionsAndActions> built in. Users who are
currently using this extension should remove it from the plugin list.
Please also remove
F<RT-Extension-AdminConditionsAndActions> from your RT
installation.

=back

=head1 UPGRADING FROM 4.4.2 AND EARLIER

=over 4

=item *

The C<MessageRef> parameter of the default callback of C</Elements/MessageBox>
now contains the reference to the message content as the name implies.

=back

=head1 UPGRADING FROM 4.4.4 AND EARLIER

=over 4

=item * Privacy Menu in Query Builder

On the Query Builder, the Privacy menu loads groups you are a member of as potential
places to save searches. However, it previously did not confirm the current user
had the EditSavedSearches right, so you might try to save a search with a group
and receive an error until that right was granted.

This has been fixed, so now groups load only if you have rights to create the
search via EditSavedSearches on the group or globally. This may change the groups
that appear in the Privacy menu, but shouldn't change functionality since users
without the correct rights were unable to create searches.

=item * AttachmentListCount Default

The C<AttachmentListCount> configuration option now defaults to 5.
To restore the previous configuration and show all attachments,
add the following to your C<RT_SiteConfig.pm>.

    Set($AttachmentListCount, undef);

=item * User Timezone and Dates in Ticket Searches

This releases fixes an issue with how "=" in time comparisons in ticket
searches applied timezone settings. Previously, dates would be adjusted
based on the global RT timezone even if the user had a different timezone.
This has been fixed to correctly use the user's timezone.

Note that this change may modify results for some saved searches for users
with a different timezone than RT's global setting.

=item * Script to reduce records in CachedGroupMembers

When adding groups to roles on tickets, we have found that caching the
members of these added groups in the CachedGroupMembers table makes
performance worse rather than improving it. This release includes updates
to no longer add these members recursively.

If you use groups in ticket roles, it's likely your CachedGroupMembers table
has a large number of now unnecessary records and these can hurt performance.
To delete these extra records run the following script:

    /usr/share/request-tracker5/etc/upgrade/shrink-cgm-table

Depending on how many records your system has, this may take a while to run.
After you run this, you may have significantly reduced the number of records
in your CachedGroupMembers table, and may need to tell your database to
refresh indexes/statistics.

=item * Run Transaction Batch last, even in nested updates

When RT processes scrips, Batch mode scrips should always run last.
Previously, with nested updates, inner updates would run batch before
all outer updates were complete. One example of this is processing ticket
updates, the Basics update calls an inner "atomic" transaction to update
Owner, which would cause the unexpected batch run. This has been fixed
so batch runs only once for the outermost updates. All transactions
performed for that batch are available from the C<TransactionBatch> method
as expected.

=item * Ambigious Owner order by option in search replaced with Owner.Name

RT 4.4.5 adds a bunch of new order by and format options for users and roles
to the Query Builder. For example, you can order by user fields on a user
like Owner.EmailAddress, Owner.RealName, or even Owner.Organization.
As part of this change, the previous Owner entry has been renamed to
Owner.Name.

The upgrade scripts include a step to make this change in any saved searches
in the database automatically. If you have Owner as an order by field in
searches stored elsewhere or as a link, you can update to Owner.Name manually.

=item * Default "order by" field for roles in search results

Because of the new options mentioned above, when displaying role information
we also set defaults for order by behavior when you click headers on search
results to re-sort them. For example, in a list of ticket results, you can
click on Requestors to re-sort the results by that field.

Previously this order by used EmailAddress. It now defaults to Name
because we believe that is more common for sorting, but because of
the new features you can change this in your search Format.
For example, you can update the line in the Format section on
the Advanced page to look like this:

    '<small>__Requestor.EmailAddress__</small>',

That column will then specifically show EmailAddress and when you click
it will order by EmailAddress.

=back

=head1 UPGRADING FROM 4.4.5 AND EARLIER

=over 4

=item * Fix for Mason cache handling

RT 4.4.5 introduced a new button on the System Configuration page to easily
clear the mason cache, and a bug was found related to the updated
Mason cache handling. The bug is most evident in extensions like
RTIR where some links will stop working correctly. This is because
some callbacks would not get correctly re-initialized on fresh page
loads. This bug is fixed in this release.

=back

=head1 UPGRADING FROM 4.4.6 AND EARLIER

=over 4

=item * Perl module change to GraphViz2

RT 4.4.6 and earlier use the Perl GraphViz module for interfacing with the graphviz
library for generating ticket link graphs.  That module has been deprecated so
we have replaced it with the GraphViz2 module.

This is installed on Debian when installing the request-tracker5 package.

=item * MySQL 8 now supported

Starting with RT 4.4.7, RT now supports MySQL 8. Note that as part of this upgrade
you also need to update the module L<DBIx::SearchBuilder>, which is provided by
libdbix-searchbuilder-perl on Debian.

=back

=cut