Inspecting mako errors, need help

listy · 04-08-2021, 09:30 AM

I often getting errors in my custom scripts, but the stacktrace from it seems to be useless. Exmple:

invalid syntax (memory:0x7fb72f3bb190, line 19)

Code:
  File "/opt/tactic/tactic/src/pyasm/command/command.py", line 267, in execute_cmd

    ret_val = cmd.execute()

  File "/opt/tactic/tactic/src/pyasm/prod/service/api_xmlrpc.py", line 281, in execute

    self2.results = exec_meth(self, ticket, meth, args)

  File "/opt/tactic/tactic/src/pyasm/prod/service/api_xmlrpc.py", line 315, in exec_meth

    results = meth(self, ticket, *args)

  File "/opt/tactic/tactic/src/pyasm/prod/service/api_xmlrpc.py", line 5691, in execute_python_script

    Command.execute_cmd(cmd)

  File "/opt/tactic/tactic/src/pyasm/command/command.py", line 267, in execute_cmd

    ret_val = cmd.execute()

  File "/opt/tactic/tactic/src/tactic/command/python_cmd.py", line 106, in execute

    results = self.get_results()

  File "/opt/tactic/tactic/src/tactic/command/python_cmd.py", line 88, in get_results

    print(exceptions.text_error_template().render())

  File "/opt/tactic/tactic/3rd_party/common/site-packages/mako/exceptions.py", line 236, in text_error_template

    return mako.template.Template(r"""

  File "/opt/tactic/tactic/3rd_party/common/site-packages/mako/template.py", line 320, in __init__

    (code, module) = _compile_text(self, text, filename)

  File "/opt/tactic/tactic/3rd_party/common/site-packages/mako/template.py", line 711, in _compile_text

    code = compile(source, cid, 'exec')

Is there a way to get any info about the place of custom script code where error apears?

***remkonoteboom*** · (This post was last modified: 04-10-2021, 03:14 PM by remkonoteboom.)

This error is coming from mako itself.

Code:
85        except Exception as e:

86            print(e)

87            print("Error in Mako code: ")

88            print(exceptions.text_error_template().render())

Which is kind of annoying because it seems that error display in mako that is raising the exception. I am not sure if there is a way to catch this if mako itself is failing to print it's own error.

listy · 04-12-2021, 09:47 AM

So, this means a can't get proper stacktrace at all?

***remkonoteboom*** · 04-15-2021, 10:50 PM

To be honest, I don't know because it appears to be a Mako bug. The code I posted above is supposed to be the correct way to extract the error from Mako.

88 print(exceptions.text_error_template().render())

This is the correct way to print out the error for a Mako template, however, render function appears to cause a stack trace. I can try and upgrade make to a later version. In my local environment, I installed this:
https://pypi.org/project/Mako/#files
and am checking it out.

I found that the Custom Widgets do actually print out the line number such as:

Reported Error: "(SyntaxError) invalid syntax (, line 16)

I am not sure why your error does not print out the line number.

listy · 04-18-2021, 06:18 PM

Ok, i found this particular error in stdout.log:

Code:
timestamp:  2021-04-07 23:04:30

method:  execute_python_script

ticket:  cfdf82f013f039f8e81

Mismatch checkin_type!

This path [/opt/tactic/assets/dolly3d/source_images/objects/equipment/Vendinng_Egg_Machine/versions/screen_normal_v001.tif] already exists

Error in Mako code: 

Error:  invalid syntax (memory:0x7f3bdc6e18e0, line 19)

--------------------------------------------------

  File "/opt/tactic/tactic/src/pyasm/command/command.py", line 267, in execute_cmd

    ret_val = cmd.execute()

  File "/opt/tactic/tactic/src/pyasm/prod/service/api_xmlrpc.py", line 281, in execute

    self2.results = exec_meth(self, ticket, meth, args)

  File "/opt/tactic/tactic/src/pyasm/prod/service/api_xmlrpc.py", line 315, in exec_meth

    results = meth(self, ticket, *args)

  File "/opt/tactic/tactic/src/pyasm/prod/service/api_xmlrpc.py", line 5691, in execute_python_script

    Command.execute_cmd(cmd)

  File "/opt/tactic/tactic/src/pyasm/command/command.py", line 267, in execute_cmd

    ret_val = cmd.execute()

  File "/opt/tactic/tactic/src/tactic/command/python_cmd.py", line 106, in execute

    results = self.get_results()

  File "/opt/tactic/tactic/src/tactic/command/python_cmd.py", line 88, in get_results

    print(exceptions.text_error_template().render())

  File "/opt/tactic/tactic/3rd_party/common/site-packages/mako/exceptions.py", line 236, in text_error_template

    return mako.template.Template(r"""

  File "/opt/tactic/tactic/3rd_party/common/site-packages/mako/template.py", line 320, in __init__

    (code, module) = _compile_text(self, text, filename)

  File "/opt/tactic/tactic/3rd_party/common/site-packages/mako/template.py", line 711, in _compile_text

    code = compile(source, cid, 'exec')

(   'dolly3d/texture',

    [   ['name', 'surpriseEgg_height'],

        ['assets_category_code', 'objects/equipment'],

        ['sub_type', 'surpriseEgg']],

    [],

    [],

    False,

    None,

    None,

    False,

    None,

    False,

    None)

Duration: 0.005 seconds (request_id: 140218107676416 - #0000082)

Memory: 215508 KB

Increment: 0 KB

This is due to strange "already exists" when uploading files.

The part of code i am executing during checkin:

Code:
def create_snapshot_extended(search_key, context, project_code=None, snapshot_type=None, is_revision=False, is_latest=True, is_current=False, description=None, version=None, level_key=None, update_versionless=True, only_versionless=False, keep_file_name=True, repo_name=None, files_info=None, mode=None, create_icon=True):

    import os

    import shutil

    import json

    from pyasm.biz import Snapshot, IconCreator

    from pyasm.checkin import FileAppendCheckin

    from pyasm.search import Search

    from pyasm.common import Environment

    if project_code:

        server.set_project(project_code)

    # mode = 'local'

    api = server.server

    sobject = api._get_sobjects(search_key)[0]

    # get the level object

    if level_key:

        levels = api._get_sobjects(level_key)

        level = levels[0]

        level_type = level.get_search_type()

        level_id = level.get_id()

    else:

        level_type = None

        level_id = None

    if not description:

        description = 'No description'

    if not snapshot_type:

        snapshot_type = 'file'

    files_info = json.loads(files_info, strict=False)

    def get_max_version(context, search_key):

        # faster way to get max snapshot version

        search = Search('sthpw/snapshot')

        search_type, search_code = server.split_search_key(search_key)

        search.add_op_filters([('context', context), ('search_code', search_code), ('search_type', search_type)])

        snaps = search.get_sobjects()

        versions = []

        for sn in snaps:

            versions.append(sn.get_attr_value('version'))

        if versions:

            return max(versions)

    if not version:

        ver = get_max_version(context=context, search_key=search_key)

        if ver is not None:

            if is_revision:

                version = int(ver)

            else:

                version = int(ver) + 1

        else:

            version = 1

    snapshot = Snapshot.create(sobject, snapshot_type=snapshot_type, context=context, description=description, is_revision=is_revision, is_latest=is_latest, is_current=is_current, level_type=level_type, level_id=level_id, commit=False, version=version)

    if repo_name:

        snapshot.set_value('repo', repo_name)

    if is_latest:

        snapshot.set_value('is_latest', 1)

    if is_current:

        snapshot.set_value('is_current', 1)

    if context.startswith('icon'):

        # This is for TACTIC 4.8 >

        update_versionless = False

    if is_revision:

        snapshot_code = server.eval("@GET(sthpw/snapshot['version', {0}].code)".format(version),

                                    search_keys=[search_key], single=True)

        revision = server.eval("@MAX(sthpw/snapshot.revision)",

                               search_keys=['sthpw/snapshot?code={0}'.format(snapshot_code)])

        snapshot.set_value('version', version)

        snapshot.set_value('revision', revision + 1)

    if mode == 'upload':

        checkin_mode = 'uploaded'

        # SOME KIND OF HACK!

        if is_revision:

            checkin_mode = 'preallocate'

        env = Environment.get()

        lib_dir = env.get_upload_dir()

        file_paths = []

        original_files_paths = []

        file_types = []

        for metadata, types in zip(files_info['version_metadata'], files_info['files_types']):

            original_files_paths.append('{0}/{1}.{2}'.format(lib_dir, metadata['filename'], metadata['new_file_ext']))

            file_paths.append('{0}/{1}.{2}'.format(lib_dir, metadata['new_filename'], metadata['new_file_ext']))

            file_types.append(types)

        # generating previews if its not explicitly passed

        if any(i in file_types for i in ['web', 'icon']):

            create_icon = False

        for original_file in file_paths[:]:

            # if this is a file, then try to create an icon

            if os.path.isfile(original_file) and create_icon:

                icon_creator = IconCreator(original_file)

                icon_creator.execute()

                web_path = icon_creator.get_web_path()

                icon_path = icon_creator.get_icon_path()

                # If this is pure icon context, then don't check in icon

                # as the main file.

                if context == 'icon':

                    if web_path:

                        shutil.copy(web_path, original_file)

                    elif icon_path:

                        shutil.copy(icon_path, original_file)

                # If web file is not generated and icon is, use original as web.

                if icon_path and not web_path:

                    base, ext = os.path.splitext(original_file)

                    web_path = "%s_web.%s" % (base, ext)

                    shutil.copy(original_file, web_path)

                if web_path:

                    file_paths.append(web_path)

                    file_types.append('web')

                    original_files_paths.append(web_path)

                    files_info['file_sizes'].append(65536)

                    files_info['version_metadata'].append('')

                    files_info['versionless_metadata'].append('')

                if icon_path:

                    file_paths.append(icon_path)

                    file_types.append('icon')

                    original_files_paths.append(icon_path)

                    files_info['file_sizes'].append(65536)

                    files_info['version_metadata'].append('')

                    files_info['versionless_metadata'].append('')

        snapshot.commit(triggers=True, log_transaction=True)

        # we keep file name as we already got name from virtual snapshot

        checkin = FileAppendCheckin(snapshot.get_code(), file_paths, file_types,

                                    keep_file_name=True, mode=checkin_mode, source_paths=file_paths,

                                    checkin_type='auto', do_update_versionless=False)

        checkin.file_sizes = files_info['file_sizes']

        checkin.execute()

        files_list = checkin.get_file_objects()

        for i, fl in enumerate(files_list):

            fl.set_value(name='source_path', value=original_files_paths[i])

            fl.set_value(name='metadata', value=json.dumps(files_info['version_metadata'][i], separators=(',', ':')))

            fl.commit(triggers=False, log_transaction=False)

        if update_versionless:

            snapshot.update_versionless('latest', sobject=sobject, checkin_type='strict')

            versionless_snapshot = snapshot.get_by_sobjects([sobject], context, version=-1)

            if repo_name:

                versionless_snapshot[0].set_value('repo', repo_name)

            versionless_snapshot[0].set_value('login', snapshot.get_attr_value('login'))

            versionless_snapshot[0].set_value('timestamp', snapshot.get_attr_value('timestamp'))

            versionless_snapshot[0].set_value('description', description)

            versionless_snapshot[0].commit(triggers=False, log_transaction=False)

            file_objects = versionless_snapshot[0].get_all_file_objects()

            for i, file_object in enumerate(file_objects):

                file_object.set_value(name='project_code', value=snapshot.get_project_code())

                file_object.set_value(name='metadata',

                                      value=json.dumps(files_info['versionless_metadata'][i], separators=(',', ':')))

                file_object.set_value(name='source_path', value=original_files_paths[i])

                file_object.commit(triggers=False, log_transaction=False)

    elif mode == 'inplace':

        if only_versionless:

            snapshot.set_value('version', -1)

            snapshot.set_value('is_current', 1)

            snapshot.set_value('is_latest', 1)

            update_versionless = False

            existing_versionless_snapshot = snapshot.get_by_sobjects([sobject], context, version=-1)

            if existing_versionless_snapshot:

                from tactic.ui.tools import DeleteCmd

                cmd = DeleteCmd(sobject=existing_versionless_snapshot[0], auto_discover=True)

                cmd.execute()

            snapshot.commit(triggers=True, log_transaction=True)

            checkin = FileAppendCheckin(snapshot.get_code(), files_info['versionless_files'], files_info['files_types'],

                                        keep_file_name=True, mode=mode, source_paths=files_info['versionless_files'],

                                        checkin_type='auto', do_update_versionless=False)

            checkin.execute()

            files_list = checkin.get_file_objects()

            for i, fl in enumerate(files_list):

                fl.set_value(name='st_size', value=files_info['file_sizes'][i])

                fl.set_value(name='relative_dir', value=files_info['versionless_files_paths'][i])

                fl.set_value(name='metadata', value=json.dumps(files_info['versionless_metadata'][i], separators=(',', ':')))

                fl.commit(triggers=False, log_transaction=False)

        else:

            snapshot.commit(triggers=True, log_transaction=True)

            # we keep file name as we already got name from virtual snapshot

            checkin = FileAppendCheckin(snapshot.get_code(), files_info['version_files'], files_info['files_types'],

                                        keep_file_name=True, mode=mode, source_paths=files_info['version_files'],

                                        checkin_type='auto', do_update_versionless=False)

            checkin.execute()

            files_list = checkin.get_file_objects()

            for i, fl in enumerate(files_list):

                fl.set_value(name='st_size', value=files_info['file_sizes'][i])

                fl.set_value(name='relative_dir', value=files_info['version_files_paths'][i])

                fl.set_value(name='metadata', value=json.dumps(files_info['version_metadata'][i], separators=(',', ':')))

                fl.commit(triggers=False, log_transaction=False)

        if update_versionless:

            existing_versionless_snapshot = snapshot.get_by_sobjects([sobject], context, version=-1)

            if existing_versionless_snapshot:

                from tactic.ui.tools import DeleteCmd

                cmd = DeleteCmd(sobject=existing_versionless_snapshot[0], auto_discover=True)

                cmd.execute()

            versionless = Snapshot.create(sobject, snapshot_type=snapshot_type, context=context, description=description,

                                       is_revision=is_revision, is_latest=is_latest, is_current=is_current,

                                       level_type=level_type, level_id=level_id, commit=False, version=version)

            if repo_name:

                versionless.set_value('repo', repo_name)

                versionless.set_value('version', -1)

                # snapshot.set_value('is_current', 1)

                versionless.set_value('is_latest', 1)

            versionless.commit(triggers=False, log_transaction=False)

            checkin = FileAppendCheckin(versionless.get_code(), files_info['versionless_files'], files_info['files_types'],

                                        keep_file_name=True, mode=mode, source_paths=files_info['versionless_files'],

                                        checkin_type='auto', do_update_versionless=False)

            checkin.execute()

            from pyasm.checkin import SnapshotBuilder

            builder = SnapshotBuilder()

            files_list = checkin.get_file_objects()

            for i, fl in enumerate(files_list):

                fl.set_value(name='st_size', value=files_info['file_sizes'][i])

                fl.set_value(name='relative_dir', value=files_info['versionless_files_paths'][i])

                fl.set_value(name='metadata', value=json.dumps(files_info['versionless_metadata'][i], separators=(',', ':')))

                info = {'type': files_info['files_types'][i]}

                builder.add_file(fl, info=info)

                fl.commit(triggers=False, log_transaction=False)

            builder.add_root_attr('ref_snapshot_code', snapshot.get_code())

            versionless.set_value("snapshot", builder.to_string())

            versionless.commit(triggers=False, log_transaction=False)

    snapshot_dict = api._get_sobject_dict(snapshot)

    return snapshot_dict

And, i think, the error is caused by Update Versionless.
But it is to hard to track all my errors that way