TACTIC Open Source
Inspecting mako errors, need help - Printable Version

+- TACTIC Open Source (http://forum.southpawtech.com)
+-- Forum: TACTIC Open Source (http://forum.southpawtech.com/forumdisplay.php?fid=3)
+--- Forum: TACTIC Discussion (http://forum.southpawtech.com/forumdisplay.php?fid=4)
+--- Thread: Inspecting mako errors, need help (/showthread.php?tid=209)



Inspecting mako errors, need help - listy - 04-08-2021

I often getting errors in my custom scripts, but the stacktrace from it seems to be useless. Exmple:

invalid syntax (memory:0x7fb72f3bb190, line 19)
Code:
  File "/opt/tactic/tactic/src/pyasm/command/command.py", line 267, in execute_cmd
    ret_val = cmd.execute()
  File "/opt/tactic/tactic/src/pyasm/prod/service/api_xmlrpc.py", line 281, in execute
    self2.results = exec_meth(self, ticket, meth, args)
  File "/opt/tactic/tactic/src/pyasm/prod/service/api_xmlrpc.py", line 315, in exec_meth
    results = meth(self, ticket, *args)
  File "/opt/tactic/tactic/src/pyasm/prod/service/api_xmlrpc.py", line 5691, in execute_python_script
    Command.execute_cmd(cmd)
  File "/opt/tactic/tactic/src/pyasm/command/command.py", line 267, in execute_cmd
    ret_val = cmd.execute()
  File "/opt/tactic/tactic/src/tactic/command/python_cmd.py", line 106, in execute
    results = self.get_results()
  File "/opt/tactic/tactic/src/tactic/command/python_cmd.py", line 88, in get_results
    print(exceptions.text_error_template().render())
  File "/opt/tactic/tactic/3rd_party/common/site-packages/mako/exceptions.py", line 236, in text_error_template
    return mako.template.Template(r"""
  File "/opt/tactic/tactic/3rd_party/common/site-packages/mako/template.py", line 320, in __init__
    (code, module) = _compile_text(self, text, filename)
  File "/opt/tactic/tactic/3rd_party/common/site-packages/mako/template.py", line 711, in _compile_text
    code = compile(source, cid, 'exec')
Is there a way to get any info about the place of custom script code where error apears?


RE: Inspecting mako errors, need help - remkonoteboom - 04-10-2021

This error is coming from mako itself.


Code:
85        except Exception as e:
86            print(e)
87            print("Error in Mako code: ")
88            print(exceptions.text_error_template().render())


Which is kind of annoying because it seems that error display in mako that is raising the exception.  I am not sure if there is a way to catch this if mako itself is failing to print it's own error.


RE: Inspecting mako errors, need help - listy - 04-12-2021

So, this means a can't get proper stacktrace at all?


RE: Inspecting mako errors, need help - remkonoteboom - 04-15-2021

To be honest, I don't know because it appears to be a Mako bug. The code I posted above is supposed to be the correct way to extract the error from Mako.

88 print(exceptions.text_error_template().render())

This is the correct way to print out the error for a Mako template, however, render function appears to cause a stack trace. I can try and upgrade make to a later version. In my local environment, I installed this:
https://pypi.org/project/Mako/#files
and am checking it out.

I found that the Custom Widgets do actually print out the line number such as:

Reported Error: "(SyntaxError) invalid syntax (, line 16)

I am not sure why your error does not print out the line number.


RE: Inspecting mako errors, need help - listy - 04-18-2021

Ok, i found this particular error in stdout.log:
Code:
timestamp:  2021-04-07 23:04:30
method:  execute_python_script
ticket:  cfdf82f013f039f8e81
Mismatch checkin_type!
This path [/opt/tactic/assets/dolly3d/source_images/objects/equipment/Vendinng_Egg_Machine/versions/screen_normal_v001.tif] already exists
Error in Mako code:
Error:  invalid syntax (memory:0x7f3bdc6e18e0, line 19)
--------------------------------------------------
  File "/opt/tactic/tactic/src/pyasm/command/command.py", line 267, in execute_cmd
    ret_val = cmd.execute()
  File "/opt/tactic/tactic/src/pyasm/prod/service/api_xmlrpc.py", line 281, in execute
    self2.results = exec_meth(self, ticket, meth, args)
  File "/opt/tactic/tactic/src/pyasm/prod/service/api_xmlrpc.py", line 315, in exec_meth
    results = meth(self, ticket, *args)
  File "/opt/tactic/tactic/src/pyasm/prod/service/api_xmlrpc.py", line 5691, in execute_python_script
    Command.execute_cmd(cmd)
  File "/opt/tactic/tactic/src/pyasm/command/command.py", line 267, in execute_cmd
    ret_val = cmd.execute()
  File "/opt/tactic/tactic/src/tactic/command/python_cmd.py", line 106, in execute
    results = self.get_results()
  File "/opt/tactic/tactic/src/tactic/command/python_cmd.py", line 88, in get_results
    print(exceptions.text_error_template().render())
  File "/opt/tactic/tactic/3rd_party/common/site-packages/mako/exceptions.py", line 236, in text_error_template
    return mako.template.Template(r"""
  File "/opt/tactic/tactic/3rd_party/common/site-packages/mako/template.py", line 320, in __init__
    (code, module) = _compile_text(self, text, filename)
  File "/opt/tactic/tactic/3rd_party/common/site-packages/mako/template.py", line 711, in _compile_text
    code = compile(source, cid, 'exec')

(   'dolly3d/texture',
    [   ['name', 'surpriseEgg_height'],
        ['assets_category_code', 'objects/equipment'],
        ['sub_type', 'surpriseEgg']],
    [],
    [],
    False,
    None,
    None,
    False,
    None,
    False,
    None)

Duration: 0.005 seconds (request_id: 140218107676416 - #0000082)
Memory: 215508 KB
Increment: 0 KB
This is due to strange "already exists" when uploading files.


The part of code i am executing during checkin:

Code:
def create_snapshot_extended(search_key, context, project_code=None, snapshot_type=None, is_revision=False, is_latest=True, is_current=False, description=None, version=None, level_key=None, update_versionless=True, only_versionless=False, keep_file_name=True, repo_name=None, files_info=None, mode=None, create_icon=True):
    import os
    import shutil
    import json
    from pyasm.biz import Snapshot, IconCreator
    from pyasm.checkin import FileAppendCheckin
    from pyasm.search import Search
    from pyasm.common import Environment

    if project_code:
        server.set_project(project_code)

    # mode = 'local'

    api = server.server

    sobject = api._get_sobjects(search_key)[0]

    # get the level object
    if level_key:
        levels = api._get_sobjects(level_key)
        level = levels[0]
        level_type = level.get_search_type()
        level_id = level.get_id()
    else:
        level_type = None
        level_id = None

    if not description:
        description = 'No description'
    if not snapshot_type:
        snapshot_type = 'file'

    files_info = json.loads(files_info, strict=False)

    def get_max_version(context, search_key):
        # faster way to get max snapshot version
        search = Search('sthpw/snapshot')
        search_type, search_code = server.split_search_key(search_key)
        search.add_op_filters([('context', context), ('search_code', search_code), ('search_type', search_type)])
        snaps = search.get_sobjects()
        versions = []
        for sn in snaps:
            versions.append(sn.get_attr_value('version'))

        if versions:
            return max(versions)

    if not version:
        ver = get_max_version(context=context, search_key=search_key)
        if ver is not None:
            if is_revision:
                version = int(ver)
            else:
                version = int(ver) + 1
        else:
            version = 1

    snapshot = Snapshot.create(sobject, snapshot_type=snapshot_type, context=context, description=description, is_revision=is_revision, is_latest=is_latest, is_current=is_current, level_type=level_type, level_id=level_id, commit=False, version=version)

    if repo_name:
        snapshot.set_value('repo', repo_name)
    if is_latest:
        snapshot.set_value('is_latest', 1)
    if is_current:
        snapshot.set_value('is_current', 1)

    if context.startswith('icon'):
        # This is for TACTIC 4.8 >
        update_versionless = False

    if is_revision:
        snapshot_code = server.eval("@GET(sthpw/snapshot['version', {0}].code)".format(version),
                                    search_keys=[search_key], single=True)
        revision = server.eval("@MAX(sthpw/snapshot.revision)",
                               search_keys=['sthpw/snapshot?code={0}'.format(snapshot_code)])

        snapshot.set_value('version', version)
        snapshot.set_value('revision', revision + 1)

    if mode == 'upload':
        checkin_mode = 'uploaded'

        # SOME KIND OF HACK!
        if is_revision:
            checkin_mode = 'preallocate'

        env = Environment.get()
        lib_dir = env.get_upload_dir()

        file_paths = []
        original_files_paths = []
        file_types = []

        for metadata, types in zip(files_info['version_metadata'], files_info['files_types']):
            original_files_paths.append('{0}/{1}.{2}'.format(lib_dir, metadata['filename'], metadata['new_file_ext']))
            file_paths.append('{0}/{1}.{2}'.format(lib_dir, metadata['new_filename'], metadata['new_file_ext']))
            file_types.append(types)

        # generating previews if its not explicitly passed
        if any(i in file_types for i in ['web', 'icon']):
            create_icon = False

        for original_file in file_paths[:]:
            # if this is a file, then try to create an icon
            if os.path.isfile(original_file) and create_icon:
                icon_creator = IconCreator(original_file)
                icon_creator.execute()

                web_path = icon_creator.get_web_path()
                icon_path = icon_creator.get_icon_path()

                # If this is pure icon context, then don't check in icon
                # as the main file.
                if context == 'icon':
                    if web_path:
                        shutil.copy(web_path, original_file)
                    elif icon_path:
                        shutil.copy(icon_path, original_file)

                # If web file is not generated and icon is, use original as web.
                if icon_path and not web_path:
                    base, ext = os.path.splitext(original_file)
                    web_path = "%s_web.%s" % (base, ext)
                    shutil.copy(original_file, web_path)

                if web_path:
                    file_paths.append(web_path)
                    file_types.append('web')
                    original_files_paths.append(web_path)
                    files_info['file_sizes'].append(65536)
                    files_info['version_metadata'].append('')
                    files_info['versionless_metadata'].append('')

                if icon_path:
                    file_paths.append(icon_path)
                    file_types.append('icon')
                    original_files_paths.append(icon_path)
                    files_info['file_sizes'].append(65536)
                    files_info['version_metadata'].append('')
                    files_info['versionless_metadata'].append('')

        snapshot.commit(triggers=True, log_transaction=True)

        # we keep file name as we already got name from virtual snapshot
        checkin = FileAppendCheckin(snapshot.get_code(), file_paths, file_types,
                                    keep_file_name=True, mode=checkin_mode, source_paths=file_paths,
                                    checkin_type='auto', do_update_versionless=False)

        checkin.file_sizes = files_info['file_sizes']
        checkin.execute()
        files_list = checkin.get_file_objects()

        for i, fl in enumerate(files_list):
            fl.set_value(name='source_path', value=original_files_paths[i])
            fl.set_value(name='metadata', value=json.dumps(files_info['version_metadata'][i], separators=(',', ':')))
            fl.commit(triggers=False, log_transaction=False)

        if update_versionless:

            snapshot.update_versionless('latest', sobject=sobject, checkin_type='strict')
            versionless_snapshot = snapshot.get_by_sobjects([sobject], context, version=-1)

            if repo_name:
                versionless_snapshot[0].set_value('repo', repo_name)
            versionless_snapshot[0].set_value('login', snapshot.get_attr_value('login'))
            versionless_snapshot[0].set_value('timestamp', snapshot.get_attr_value('timestamp'))
            versionless_snapshot[0].set_value('description', description)
            versionless_snapshot[0].commit(triggers=False, log_transaction=False)
            file_objects = versionless_snapshot[0].get_all_file_objects()

            for i, file_object in enumerate(file_objects):
                file_object.set_value(name='project_code', value=snapshot.get_project_code())
                file_object.set_value(name='metadata',
                                      value=json.dumps(files_info['versionless_metadata'][i], separators=(',', ':')))
                file_object.set_value(name='source_path', value=original_files_paths[i])
                file_object.commit(triggers=False, log_transaction=False)
    elif mode == 'inplace':

        if only_versionless:
            snapshot.set_value('version', -1)
            snapshot.set_value('is_current', 1)
            snapshot.set_value('is_latest', 1)
            update_versionless = False
            existing_versionless_snapshot = snapshot.get_by_sobjects([sobject], context, version=-1)
            if existing_versionless_snapshot:
                from tactic.ui.tools import DeleteCmd
                cmd = DeleteCmd(sobject=existing_versionless_snapshot[0], auto_discover=True)
                cmd.execute()

            snapshot.commit(triggers=True, log_transaction=True)

            checkin = FileAppendCheckin(snapshot.get_code(), files_info['versionless_files'], files_info['files_types'],
                                        keep_file_name=True, mode=mode, source_paths=files_info['versionless_files'],
                                        checkin_type='auto', do_update_versionless=False)
            checkin.execute()

            files_list = checkin.get_file_objects()

            for i, fl in enumerate(files_list):
                fl.set_value(name='st_size', value=files_info['file_sizes'][i])
                fl.set_value(name='relative_dir', value=files_info['versionless_files_paths'][i])
                fl.set_value(name='metadata', value=json.dumps(files_info['versionless_metadata'][i], separators=(',', ':')))
                fl.commit(triggers=False, log_transaction=False)

        else:

            snapshot.commit(triggers=True, log_transaction=True)
            # we keep file name as we already got name from virtual snapshot
            checkin = FileAppendCheckin(snapshot.get_code(), files_info['version_files'], files_info['files_types'],
                                        keep_file_name=True, mode=mode, source_paths=files_info['version_files'],
                                        checkin_type='auto', do_update_versionless=False)
            checkin.execute()

            files_list = checkin.get_file_objects()

            for i, fl in enumerate(files_list):
                fl.set_value(name='st_size', value=files_info['file_sizes'][i])
                fl.set_value(name='relative_dir', value=files_info['version_files_paths'][i])
                fl.set_value(name='metadata', value=json.dumps(files_info['version_metadata'][i], separators=(',', ':')))
                fl.commit(triggers=False, log_transaction=False)

        if update_versionless:
            existing_versionless_snapshot = snapshot.get_by_sobjects([sobject], context, version=-1)
            if existing_versionless_snapshot:
                from tactic.ui.tools import DeleteCmd
                cmd = DeleteCmd(sobject=existing_versionless_snapshot[0], auto_discover=True)
                cmd.execute()

            versionless = Snapshot.create(sobject, snapshot_type=snapshot_type, context=context, description=description,
                                       is_revision=is_revision, is_latest=is_latest, is_current=is_current,
                                       level_type=level_type, level_id=level_id, commit=False, version=version)

            if repo_name:
                versionless.set_value('repo', repo_name)
                versionless.set_value('version', -1)
                # snapshot.set_value('is_current', 1)
                versionless.set_value('is_latest', 1)

            versionless.commit(triggers=False, log_transaction=False)

            checkin = FileAppendCheckin(versionless.get_code(), files_info['versionless_files'], files_info['files_types'],
                                        keep_file_name=True, mode=mode, source_paths=files_info['versionless_files'],
                                        checkin_type='auto', do_update_versionless=False)
            checkin.execute()

            from pyasm.checkin import SnapshotBuilder
            builder = SnapshotBuilder()

            files_list = checkin.get_file_objects()

            for i, fl in enumerate(files_list):
                fl.set_value(name='st_size', value=files_info['file_sizes'][i])
                fl.set_value(name='relative_dir', value=files_info['versionless_files_paths'][i])
                fl.set_value(name='metadata', value=json.dumps(files_info['versionless_metadata'][i], separators=(',', ':')))
                info = {'type': files_info['files_types'][i]}
                builder.add_file(fl, info=info)
                fl.commit(triggers=False, log_transaction=False)

            builder.add_root_attr('ref_snapshot_code', snapshot.get_code())

            versionless.set_value("snapshot", builder.to_string())

            versionless.commit(triggers=False, log_transaction=False)

    snapshot_dict = api._get_sobject_dict(snapshot)

    return snapshot_dict

And, i think, the error is caused by Update Versionless. 
But it is to hard to track all my errors that way